Senior Data Engineer
Posted on Nov 1, 2020 by Gazelle Global Consulting
Senior Data Engineer: BANKING, BRUSSELS
- Extract structured and unstructured data from the source systems (relational databases, data warehouses, document repositories, file systems,), prepare such data (cleanse, re-structure, aggregate,) and load them onto Hadoop.
- Actively support the reporting teams in the data exploration and data preparation phases.
- Implement data quality controls and where data quality issues are detected, liaise with the data supplier for joint root cause analysis
- Be able to autonomously design data pipelines, develop them and prepare the launch activities
- Properly document your code, share and transfer your knowledge with the rest of the team to ensure a smooth transition into maintenance and support of production applications
- Experience with analysis and creation of data pipelines, data architecture, ETL/ELT development and with processing structured and unstructured data
- Proven experience with using data stored in RDBMSs and experience or good understanding of NoSQL databases
- Ability to write performant Scala code and SQL statement
- Knowledge of Python and Spark
- Understanding of the Hadoop ecosystem including Hadoop file formats like Parquet and ORC
- Experience with open source technologies used in Data Analytics like Spark, Pig, Hive, HBase, Kafka,
Interview slots as soon as next week Monday to Wednesday so if you interested please do not hesitate to apply!