Python/Pyspark Data Engineer

Looking for an experienced Python/Pyspark Data Engineer, to support their projects based in Zurich

The Role:

• Our client is looking for a data engineer who will help build new or improve existing data pipelines.


• You should be comfortable working with large or fast moving data, have a solid understanding of distributed processing frameworks, and a software engineering mindset


• Role involves knowing and coding in big data, transforming data in the data pipeline, scheduling data pipelines, writing performant big data pipelines.


• It would be good if you have used Python in Spark programming, but you are not expected to code in python.

Skills and Experience required:

• Over all 7 to 12 years of IT experience. Extensive experience in Big Data, Analytics, ETL technologies


• Minimum 2 to 4 years of experience in Spark programming using either Python/Scala/Java.


• Application Development background on big data along with knowledge of Analytics libraries and big data computing libraries


• Hands on experience in coding, designing and development of complex data pipelines using big data technologies


• Experience in developing applications on Big Data. Design and build highly scalable data pipelines


• Experience in Python, SQL Database, Spark, non-relational databases


• Responsible to ingest data from files, streams and databases. Process the data using Spark, Python


• Develop programs in PySpark as part of data cleaning and processing


• Responsible to design and develop distributed, high volume, high velocity multi-threaded event processing systems


• Develop efficient software code for multiple use cases leveraging Python and Big Data technologies for various use cases built on the platform


• Provide high operational excellence guaranteeing high availability and platform stability


• Implement scalable solutions to meet the ever-increasing data volumes, using big data/Palantir technologies Pyspark, any Cloud computing etc.


• Knowledge of Palantir would be added advantage


• Individual who can work under their own direction towards agreed targets/goals and with creative approach to work


• Intuitive individual with an ability to manage change and proven time management


• Proven interpersonal skills while contributing to team effort by accomplishing related results as needed

Nice To Have Skills:

• Experience in Palantir


• Knowledge of CI/CD Pipelines, Git, Jenkins


• Have worked with large datasets


• Proficient reading and understanding enterprise-grade PySpark code