Senior Data Engineer

Venesky-Brown's client, a public sector organisation in Cardiff, is currently looking to recruit 2 x Senior Data Engineers for initial 12 week contracts with potential to extend on a rate of £167.07/day (Inside of IR35). These roles will be working from home until further notice.

Responsibilities:

- Design and build scalable data pipelines for batch and Real Time processing using cloud-based tools.
- Manage cloud data architectures (data lakes, data warehouses) and ensure performance, scalability, and security.
- Implement data quality frameworks, data transformations, and data models for analytical purposes.
- Optimize cloud infrastructure for cost, performance, and resource usage.
- Collaborate cross-functionally with data scientists, analysts, and other stakeholders to meet business requirements.
- Maintain documentation on data pipelines, architectures, and systems.
- Drive innovation in data processing techniques and technologies, staying current with cloud and big data trends.

Essential Skills:

- Bachelor's Degree in Computer Science, Engineering, Information Technology, or related field.
- Experience in cloud platform as a service (PaaS).
- Ability to communicate between technical and non-technical stakeholders, supporting multidisciplinary discussions.
- Ability to undertake data profiling and source system analysis, sharing clear insights with colleagues.
- Can design, build and test complex large scale data products.
- Ability to build teams to complete data integration services.
- Can understand the impact on the organisation of emerging trends in data tools, analysis techniques and data usage.
- Ability to select and implement appropriate technologies to deliver resilient, scalable and future-proofed data solutions.
- Can produce relevant data models, explaining which model to use for which purpose.
- Ability to understand industry-recognised data modelling patterns and standards and comparison of different data models.
- Ability to respond to problems in databases, data processes, data products and services as they occur, determining the appropriate Remedy and initiating actions to resolve them.
- Can use agreed standards and tools to design, code, test, correct and document moderate-to-complex programs and scripts, collaborating with others as appropriate.
- Have an understanding of the core technical concepts related to the role and how to apply these with guidance.
- Ability to review requirements, define test conditions, identify issues and risks and analyse and report test activities.

Desirable Skills:

- Master's Degree in Computer Science, Engineering, Information Technology, or related field.
- Cloud Platform Certification (AWS, Azure, or Google Cloud Platform)
- AWS Certified Data Analytics - Specialty
- Google Professional Data Engineer
- Microsoft Certified: Azure Data Engineer
- Data Engineering Certifications (eg, Cloudera, Databricks)
- Data-related Certifications (eg, Microsoft Certified: Data Scientist Associate, Databricks Certified Data Engineer)
- Experience with Google Cloud Platform for data and analytics.
- Ability to design an appropriate metadata repository, suggest improvements for existing repositories, understand a range of tools to managing metadata and advise team members on metadata management.

If you would like to hear more about these opportunities please get in touch.