Azure Data Engineer
Posted on Aug 19, 2019 by Global Resourcing
Our client are looking for experienced software developers who are data savvy and have good interpersonal skills to join their truly diverse DevOps team. The teams aim is reinventing the IT landscape of Operations department within the bank, using Azure technology stack. They are acting as the pathfinder for the rest of DevOps teams, especially in the area putting Artificial Intelligent (AI) models in production for fully automating highly sensitive and error-prone Operational processes. On Azure, the team perform a wide range of DevOps data engineering manoeuvres while implementing GDPR, security, E2E monitoring and compliance to enable the-state-of-the-art enterprise reporting, putting AI models in production, and interoperations between these two areas. As a member of the team, your core responsibility will be to build such a solid data foundation to enable the banks operations meets their data-driven ambitious goals.
Some important aspects of the job are listed below:
- Build a solid, robust and scalable data foundation to be used for management reporting and advanced analytics.
- Design, implement, and continuously expand data pipelines.
- Work together with our Data Scientist in building and rolling out machine learning models to production.
Required Skills & Qualifications
- 8+ years of recent experience in object-oriented software development in Python (or in combination with any other object-oriented language such as Java)
- A thorough mastery of SQL.
- Hands-on expertise in Spark, PySpark.
- Demonstrable experience with Bash, Windows PowerShell, Azure Cloudshell or another Shell Scripting language.
- Experience in designing and building big data pipelines, data warehouses, or other (preferably distributed) data stores.
- Fluency in English both written and verbal.
- A university degree in software engineering or a technical field.
Nice to Have:
- Good understanding of basic analytics and machine learning concepts.
- Able to architect, design and build reusable components, frameworks and libraries at scale to support data & analytics use cases.
- Able to drive collaborative reviews of design, code, test plans and dataset implementation performed by other data engineers/data scientists.
- Experience working with public cloud solutions (preferably Azure) and their respective Advanced Analytics and Database offerings.
- Knowledge and experience with (automated) testing concepts/TDD.
- Extensive experience in data vault, dimensional Datamodelling and data warehousing architecting in general.
- Experience building and maintaining CICD pipelines preferably in Azure DevOps.
- Experience with the Agile way of working.
- Eager to innovate, experiment and advice on applying new technologies.
- Team player with good communication skills.