Senior Data Engineer
Posted on Feb 26, 2020 by Gazelle Global Consulting
Senior Data Engineer
One of my globally leading Financial clients in Utrecht is urgently looking for a Senior Data Engineer for the Advanced Analytics team that develops the client's data model. As a Data Engineer you ensure that data sources are properly accessed, made available via pipelines and monitored for quality. The data pipelines are used by the Data Scientists in setting up new Analytics solutions, and also to make these solutions available to end users. In the client's brand new Microsoft Azure cloud environment you help build a data foundation in which all data sources are made accessible for further analysis. You interact a lot with colleagues, offer solutions and ensure improvements. You also play a major role in setting up and managing the product CI/CD pipelines (Continuous Delivery and Deployment), within which the Analytics models are realized from development to production.
The ideal Senior Data Engineer will have the following responsibilities:
Unlocking new data sources with the help of NiFi.
Setting up data pipelines between source & analytics solution and between solution & end user.
Setting up the CI/CD pipelines for the Analytics products.
The development and design of applications for the end user.
Take shared responsibility for the sprint results.
Want to be T-shaped, for example by gaining expertise in data science or software engineering.
Proactively inform the Product Owner.
Provide solicited/unsolicited feedback to your teammates and you receive positive feedback.
Support and help your teammates, whereby the team is self-organizing.
Meticulous, persistent, analytical, supportive and relationship-oriented.
Promote an Agile and DevOps method.
The ideal Senior Data Engineer will have the following skills:
Relevant completed HBO/WO education, for example in AI, (applied) mathematics, data science or computer science.
3-7 years of experience in data engineering and/or software engineering.
Able to conceive the application architecture and to provide solicited/unsolicited advice on this.
Experience with Python, SQL and PySpark.
Experience with the Agile and DevOps method.
Experience with Microsoft Azure DevOps.
Experience with technologies such as NiFi, Hadoop, Hive and Linux.
Experience with structural monitoring of data quality and determination of quality standards.
Experience with Real Time analyzes and IoT data is an advantage.
Contract duration: 12 months.
Please get in touch ASAP for further details.