• Design, develop, and maintain end-to-end data pipelines (ETL/ELT) using Spark, SQL, and related technologies.
• Implement business logic, data governance, security, and retention requirements within pipelines.
• Translate business and technical requirements into scalable data solutions and detailed designs.
• Collaborate with business stakeholders, product owners, and technical teams to address data needs and issues.
• Contribute to data architecture, ensuring scalable and high-performance system design.
• Support continuous delivery through effective governance, transparency, and communication.
• Stay up to date with emerging data technologies and support proof-of-concept initiatives.

• 5+ years of experience working with modern data technologies.
• Strong experience building data ingestion pipelines for data warehouse, data lake, or lakehouse architectures.
• Hands-on expertise with big data and open-source tools (eg, Spark, Hadoop, Hive, Kafka, dbt, ELK), ideally in a Cloudera environment.
• Proficiency in Python and strong SQL skills.
• Solid understanding of data modelling, scalable system design, and data engineering best practices.
• Experience in financial services (eg, trade life cycle, risk management).
• Familiarity with lakehouse technologies such as Apache Iceberg.
• Experience with relational and NoSQL databases (eg, Postgres, SQL Server, Cassandra).
• Exposure to workflow orchestration (Airflow, NiFi), streaming (Kafka, Spark Streaming), and CI/CD pipelines.
• Knowledge of Agile methodologies and testing practices.
• Experience with cloud platforms (Databricks, Snowflake, Azure Synapse) - nice-to-have
• Understanding of microservices and APIs (REST/SOAP) - nice-to-have
• Exposure to .NET/C# - nice-to-have
• You bring strong communication skills

We believe in equal opportunity for all and actively encourage applications from diverse backgrounds, experiences, and perspectives.

Source Group International Ltd is acting as an Employment Business in relation to this vacancy