We are looking for a Senior Data Engineer to join our team and play a key role in building and optimizing data pipelines on our Azure Databricks platform for economic data analysis. This platform is crucial for Monetary Analysis, Forecasting, and Modelling activities.

Note: This project needs Active Security clearance.

Key Responsibilities:

• Develop and optimize scalable data pipelines (ETL) on Azure Databricks
• Implement data transformations using PySpark/Scala
• Ensure data quality, governance, and security (Azure Purview)
• Work with Azure data services (Data Factory, Blob Storage, SQL DB)
• Collaborate with data scientists, economists, and DevOps teams
• Automate deployments with CI/CD pipelines and DevOps best practices

Required Skills & Experience:

• 10+ years in Data Engineering, with 3+ years on Azure Databricks
• Strong expertise in Python, Spark (PySpark), Scala, SQL
• Hands-on experience with Azure services (Data Factory, Blob Storage, SQL)
• Knowledge of data warehousing, data modelling & integration patterns
• Experience with DevOps tools (Azure DevOps, Docker, Kubernetes, CI/CD)
• Strong problem-solving & stakeholder communication skills
• Azure Data Engineer Associate certification (preferred)