This Job Vacancy has Expired!

Senior Big Data Engineer - Utrecht, Netherlands 6 months +

Posted on Mar 20, 2020 by Next Ventures Ltd

Utrecht, Netherlands
IT
30 Mar 2020
Annual Salary
Contract/Project

Senior Big Data Engineer - Utrecht, Netherlands 6 months +

We are looking for a software developer to be responsible for delivering data to reporting, operational systems for our finance client. You will be in charge of developing complete and scalable solutions in Spark on a Cloudera platform using Azure DevOps pipelines.

The Data and Content within the organization has the responsibility to help teams with data for all IT Systems applications. The goal is to deliver data in a secure, compliant and automated way as fast as possible meeting Security DevOps principles working together with all data teams.

These teams are responsible for data storage, data processing, data flows and data provisioning. We work in DevOps teams where we work on Datamodelling, data logistics, data quality, data lake and data warehousing to adjust and maintain the data flows and data services we provide for systems and departments within the company.

For one of our teams working on the data lake we are looking for a senior big data engineer.

You will be responsible for
Development and maintenance of data pipelines mainly in Spark.
Setting up and maintain CI/CD pipelines in Azure devops.
Setting up monitoring.
Development of scalable and performant solutions based on open source technologies (Spark, Sqoop, Hive, Hbase, Oozie)
Streaming flow development with HDF (hortonworks) NiFi and Spark streaming.
Advice on and involvement in the realization of the Data Lake .
Working in a multi-disciplinary scrum team and contributing to the T-shaping in the team.
Providing input and contributing to experiments.
Keeping informed on the developments in the market.

The competences/expertise we would like you to have:
Willingness to learn and being open to new techniques.
Willingness and capability to teach and share knowledge.
Good communication skills.
Strong in collaboration.
Showing entrepreneurship.
Flexibility.
Knowledge on financial products and business processes.
Being able to work in a structured manner.
Able to advise and communicate on new solutions

The knowledge we would like you to have:
Knowledge on (Big) Data environments and tooling (Hadoop, Cloudera, Hortonworks Microsoft Azure, NiFi tooling)
Relevant HBO or WO education (Informatica, Informatiekunde of Bedrijfskundige richting)
4-6 years of experience in ICT data environment
Extensive experience in software development in an agile environment
Cloudera 5.12.1
Streaming (Kafka)
Spark
Scala
Java
Cloud platform (preferably Azure)
CI/CD pipelines (preferably Azure devops)
SQL + NoSQL (at least 2 years of experience)
Scripting languages (Python/Bash)

If you are interested, please get in touch: (see below)


Reference: 866779451

Set up alerts to get notified of new vacancies.