Lead Data Engineer - Big Data, London, Remote
Posted on Oct 5, 2021 by BlackCode
Lead Data Engineer - Big Data, London, Remote & Flexible working
Can you build data-intensive applications with Python, Java, or Scala?
Have you managed cloud-native big data environments?
Do you have experience building and maintaining data warehouses with big data technologies?
If any or all of this applies to you, you may be interested in a newly created opportunity for a Lead Data Engineer for a fast-growing team, in a greenfield environment.
Your responsibilities will include:
- Understand the needs of your internal customers and convert them to optimized and maintainable tech designs.
- Use your data engineering skills to design and build the ingestion, processing, storage and consumption system for data to enable other business units to make business and operational decisions using data.
- Maintain and operate the data platform, which many business units rely on to fulfil their service level targets.
- At least 2 years of experience designing and operating data pipelines and databases
- Proficiency in Python, Java, or Scala with a good understanding of runtime complexities
- Proficiency in database operation and optimization, including SQL optimization
- Strong understanding of and experience in big data tools such as Hadoop, Spark, Flink, Storm
- Experience in testing ETL pipelines