Lead/Data Engineer -AWS

E-Solutions IT Services UK Ltd

Posted on Feb 2, 2024 by E-Solutions IT Services UK Ltd
Milton Keynes, Buckinghamshire, United Kingdom
IT
Immediate Start
Annual Salary
Full-Time

Job Description: (100% onsite permanent role)

As a Data Engineer with AWS experience you will play a crucial role in the design, development, and maintenance of our data infrastructure. Your work will empower data-driven decision-making and contribute to the success of our data-driven initiatives.

Key Responsibilities:

  • Data Integration: Develop and maintain data pipelines to extract, transform, and load (ETL) data from various sources into AWS data stores for both batch and streaming data ingestion.
  • AWS Expertise: Utilize your expertise in AWS services such as Amazon EMR, S3, AWS Glue, Amazon Redshift, AWS Lambda, and more to build and optimize data solutions.
  • Data Modeling: Design and implement data models to support analytical and reporting needs, ensuring data accuracy and performance.
  • Data Quality: Implement data quality and data governance best practices to maintain data integrity.
  • Performance Optimization: Identify and resolve performance bottlenecks in data pipelines and storage solutions to ensure optimal performance.
  • Documentation: Create and maintain comprehensive documentation for data pipelines, architecture, and best practices.
  • Collaboration: Collaborate with cross-functional teams, including data scientists and analysts, to understand data requirements and deliver high-quality data solutions.
  • Automation: Implement automation processes and best practices to streamline data workflows and reduce manual interventions.

Must have: AWS, ETL, EMR, GLUE, Spark/Scala, Java, Python,

Good to have: Cloudera - Spark, Hive, Impala, HDFS, Informatica PowerCenter, Informatica DQ/DG, Snowflake Erwin

Qualifications:

  • Bachelor's or Master's degree in Computer Science, Data Engineering, or a related field.
  • Experience in data engineering, including working with AWS services.
  • Proficiency in AWS services like S3, Glue, Redshift, Lambda, and EMR.
  • Knowledge on Cloudera based hadoop is a plus.
  • Strong ETL development skills and experience with data integration tools.
  • Knowledge of data modelling, data warehousing, and data transformation techniques.
  • Familiarity with data quality and data governance principles.
  • Strong problem-solving and troubleshooting skills.
  • Excellent communication and teamwork skills, with the ability to collaborate with technical and non-technical stakeholders.
  • Knowledge of best practices in data engineering, scalability, and performance optimization.
  • Experience with version control systems and DevOps practices is a plus.

Reference: 2710578111

https://jobs.careeraddict.com/post/87726356

This Job Vacancy has Expired!

E-Solutions IT Services UK Ltd

Lead/Data Engineer -AWS

E-Solutions IT Services UK Ltd

Posted on Feb 2, 2024 by E-Solutions IT Services UK Ltd

Milton Keynes, Buckinghamshire, United Kingdom
IT
Immediate Start
Annual Salary
Full-Time

Job Description: (100% onsite permanent role)

As a Data Engineer with AWS experience you will play a crucial role in the design, development, and maintenance of our data infrastructure. Your work will empower data-driven decision-making and contribute to the success of our data-driven initiatives.

Key Responsibilities:

  • Data Integration: Develop and maintain data pipelines to extract, transform, and load (ETL) data from various sources into AWS data stores for both batch and streaming data ingestion.
  • AWS Expertise: Utilize your expertise in AWS services such as Amazon EMR, S3, AWS Glue, Amazon Redshift, AWS Lambda, and more to build and optimize data solutions.
  • Data Modeling: Design and implement data models to support analytical and reporting needs, ensuring data accuracy and performance.
  • Data Quality: Implement data quality and data governance best practices to maintain data integrity.
  • Performance Optimization: Identify and resolve performance bottlenecks in data pipelines and storage solutions to ensure optimal performance.
  • Documentation: Create and maintain comprehensive documentation for data pipelines, architecture, and best practices.
  • Collaboration: Collaborate with cross-functional teams, including data scientists and analysts, to understand data requirements and deliver high-quality data solutions.
  • Automation: Implement automation processes and best practices to streamline data workflows and reduce manual interventions.

Must have: AWS, ETL, EMR, GLUE, Spark/Scala, Java, Python,

Good to have: Cloudera - Spark, Hive, Impala, HDFS, Informatica PowerCenter, Informatica DQ/DG, Snowflake Erwin

Qualifications:

  • Bachelor's or Master's degree in Computer Science, Data Engineering, or a related field.
  • Experience in data engineering, including working with AWS services.
  • Proficiency in AWS services like S3, Glue, Redshift, Lambda, and EMR.
  • Knowledge on Cloudera based hadoop is a plus.
  • Strong ETL development skills and experience with data integration tools.
  • Knowledge of data modelling, data warehousing, and data transformation techniques.
  • Familiarity with data quality and data governance principles.
  • Strong problem-solving and troubleshooting skills.
  • Excellent communication and teamwork skills, with the ability to collaborate with technical and non-technical stakeholders.
  • Knowledge of best practices in data engineering, scalability, and performance optimization.
  • Experience with version control systems and DevOps practices is a plus.

Reference: 2710578111

CareerAddict

Alert me to jobs like this:

Amplify your job search:

CV/résumé help

Increase interview chances with our downloads and specialist services.

CV Help

Expert career advice

Increase interview chances with our downloads and specialist services.

Visit Blog

Job compatibility

Increase interview chances with our downloads and specialist services.

Start Test

Similar Jobs

Tender Support Representative

Milton Keynes, Buckinghamshire, United Kingdom

Senior Firmware & Hardware Engineer (C & Python)

Milton Keynes, Buckinghamshire, United Kingdom

Senior SOC Analyst - Milton Keynes

Milton Keynes, Buckinghamshire, United Kingdom

Dynamics D365 CE Developer

Milton Keynes, Buckinghamshire, United Kingdom