This Job Vacancy has Expired!

Python/Pyspark Data Engineer

Connect 44 AG

Posted on Apr 5, 2021 by Connect 44 AG

Zürich, Switzerland
Immediate Start
Annual Salary

Looking for an experienced Python/Pyspark Data Engineer, to support their projects based in Zurich

The Role:

  • Our client is looking for a data engineer who will help build new or improve existing data pipelines.

  • You should be comfortable working with large or fast moving data, have a solid understanding of distributed processing frameworks, and a software engineering mindset

  • Role involves knowing and coding in big data, transforming data in the data pipeline, scheduling data pipelines, writing performant big data pipelines.

  • It would be good if you have used Python in Spark programming, but you are not expected to code in python.

Skills and Experience required:

  • Over all 7 to 12 years of IT experience. Extensive experience in Big Data, Analytics, ETL technologies

  • Minimum 2 to 4 years of experience in Spark programming using either Python/Scala/Java.

  • Application Development background on big data along with knowledge of Analytics libraries and big data computing libraries

  • Hands on experience in coding, designing and development of complex data pipelines using big data technologies

  • Experience in developing applications on Big Data. Design and build highly scalable data pipelines

  • Experience in Python, SQL Database, Spark, non-relational databases

  • Responsible to ingest data from files, streams and databases. Process the data using Spark, Python

  • Develop programs in PySpark as part of data cleaning and processing

  • Responsible to design and develop distributed, high volume, high velocity multi-threaded event processing systems

  • Develop efficient software code for multiple use cases leveraging Python and Big Data technologies for various use cases built on the platform

  • Provide high operational excellence guaranteeing high availability and platform stability

  • Implement scalable solutions to meet the ever-increasing data volumes, using big data/Palantir technologies Pyspark, any Cloud computing etc.

  • Knowledge of Palantir would be added advantage

  • Individual who can work under their own direction towards agreed targets/goals and with creative approach to work

  • Intuitive individual with an ability to manage change and proven time management

  • Proven interpersonal skills while contributing to team effort by accomplishing related results as needed

Nice To Have Skills:

  • Experience in Palantir

  • Knowledge of CI/CD Pipelines, Git, Jenkins

  • Have worked with large datasets

  • Proficient reading and understanding enterprise-grade PySpark code

Reference: 1153508789

Set up alerts to get notified of new vacancies.