Big Data Engineer

NextLink is working with top notch consulting company, who is seeking for a Big Data Engineer in Leuven, Belgium for a 12 months contract

Job Title: Big Data Engineer

Duration: 12 Months contract

Location: Leuven, Belgium

Responsibilities:

1. Designing, developing, constructing, testing and maintaining the complete data management & processing systems - Data Pipelines


2. Aggregate & Transform raw data coming from a variety of data sources to fulfil the functional & non-functional business needs - Data Transformation


3. Discovering various opportunities for data acquisitions and exploring new ways of using existing data - Data Ingestion


4. Creating of data models to reduce system complexity and hence increase efficiency and reduce cost - Data Architecture & Models


5. Performance optimization & monitoring: automating processes, optimizing data delivery & re-designing the complete architecture to improve performance


6. Proposing ways to improve data quality, reliability & efficiency of the whole system - Data Quality


7. Creating a solution by integrating a variety of programming languages and tools together - Data Value


8. As a senior, it is expected to be very well communicative and to be extravert for what concerns solutions (technical-architectural), actively helping and supporting the other team members.

Competencies

Store:

• Datamodelling


• Data Architecture


• Big Data Framework/Hadoop: HDFS, Squid, Spark, Conda, Yarn and MapReduce


• NoSQL Databases: Cassandra, Hbase, MongoDB

Access & Transport - Connectivity:

• ETL (Extract, Transform, Load): Informatica


• Big Data Framework/Hadoop: Flume & Sqoop, Yarn, Zookeeper

Enrich:

• Real Time processing framework - Apache Spark


• Big Data Framework/Hadoop: PIG, Hive


• SQL and NoSQL


• Machine learning (nice to have): Python & algorithms

Provision:

• Workflow


• Programming: Java, Python (and optionally Scala)

Development methodologies:

• Agile: Safe or Spotify


• DataOps

Ancillary capabilities:

• Very strong communication skills


• Problem Solving


• Teamwork


• Innovation