This Job Vacancy has Expired!

Data Engineer - Senior Data Engineer

Consol Partners

Posted on Jan 14, 2021 by Consol Partners

Brussel, Belgium
IT
Immediate Start
€450 - €550 Daily
Contract/Project


Data Engineer - Senior Data Engineer


We are looking for an experienced Data Engineer to help join our team for a long term project in Brussels! This role will initially start of as remote working but will move on site when the virus situation is safe to do so.


As a Data Engineer will play a key role preparing the infrastructure and data that will be used to deliver high quality products whilst also helping to design, develop and maintain data pipelines that will deliver insights.


The Role:


  • Conceive and build data architectures

  • Participate in the short/mid/long term vision of the overall system

  • Execute ETL (extract/transform/load) processes from complex and/or large data sets

  • Ensure data are easily accessible and that their exploitation is performing as requested, even in highly scalable circumstances

  • Participate to the architecture and planning of the big data platform to optimize the ecosystem's performances

  • Create large data warehouses fit for further reporting or advanced analytics

  • Collaborate with machine learning engineers for the implementation and deployment of different solutions

  • Ensure robust CI/CD processes are in place

Requirements:

  • Demonstrable experience with big data platforms (Hadoop, Cloudera, EMR, Databricks, ...)

  • Technical knowledge in:
    • Data pipeline management

    • Cluster management

    • Workflow management ( Oozie, Airflow)

    • Database management of SQL and noSQL databases

    • Large file storage (HDFS, Data Lake, S3, Blob storage,..)


  • Strong knowledge of Hadoop ecosystem: Hortonworks/Cloudera/EMR

  • Strong knowledge of Java/Scala and Python

  • Strong knowledge of Spark (Scala and Pyspark)

  • Strong knowledge of CI/CD concepts

  • Stream processing such as Kafka, Kinesis, Elisticsearch

  • Good knowledge of a cloud environment

  • High level understanding of data science concepts

  • Knowledge of Data Visualisation framework like QlikSense is a plus

This is a fantastic opportunity to work on a long term project for a major, global organisation so please apply below!


Data Engineer - Senior Data Engineer




Reference: 1060114175

Set up alerts to get notified of new vacancies.