(SC cleared) data engineer - AWS/Spark

Posted on Apr 3, 2024 by Methods Business and Digital Technology Limited
Bristol, Somerset, United Kingdom
IT
Immediate Start
£450 - £450 Daily
Contract/Project

Daily rate: £450 outside IR35

Hybrid: 2days/w in Bristol

The job:
The role will require support the lead data engineer in the analysis and process of the raw platform data and additionally working with the lead data architect to design an ETL pipeline to a cloud data platform. We are looking for a candidate who understands and knows how to code using Spark technology using pySpark as a minimum. Experience of a cloud infrastructure will be a bonus.

Additionally the role will require the support of stakeholder engagement of the source systems in various workshops to understand extant data quality issues and to play back solutions that we are developing for stakeholder endorsement. You will be supported with a Business Analysts to bring all of this together.

Responsibilities

  • Analyze and organize raw platform and data warehouse data
  • Build data systems and pipelines in AWS, Delta
  • Evaluate business needs and objectives with Data Architect
  • Interpret trends and patterns
  • Conduct complex data analysis and report on results
  • Assist in the data lineage tasks
  • Assist in the empirical data analysis of source and data warehouse data sets
  • Build prototypes
  • Combine raw information from different sources
  • Explore ways to enhance data quality and reliability using AWS tooling
  • Identify opportunities for data acquisition
  • Develop analytical tools and programs
  • Collaborate with stakeholders and architects
  • Requirements analysis with BA.

Requirements and skills

  • Previous experience as a data engineer or in a similar role
  • Technical expertise with data models, data mining, and segmentation techniques
  • Knowledge of programming languages (eg Java and Python, Scala, R, SQL)
  • Hands-on experience with SQL database design
  • Great numerical and analytical skills
  • Degree in Computer Science, IT, or similar numeric field
  • Data engineering certification (eg IBM Certified Data Engineer) is a plus

Reference: 2743280310

https://jobs.careeraddict.com/post/89590907

This Job Vacancy has Expired!

(SC cleared) data engineer - AWS/Spark

Posted on Apr 3, 2024 by Methods Business and Digital Technology Limited

Bristol, Somerset, United Kingdom
IT
Immediate Start
£450 - £450 Daily
Contract/Project

Daily rate: £450 outside IR35

Hybrid: 2days/w in Bristol

The job:
The role will require support the lead data engineer in the analysis and process of the raw platform data and additionally working with the lead data architect to design an ETL pipeline to a cloud data platform. We are looking for a candidate who understands and knows how to code using Spark technology using pySpark as a minimum. Experience of a cloud infrastructure will be a bonus.

Additionally the role will require the support of stakeholder engagement of the source systems in various workshops to understand extant data quality issues and to play back solutions that we are developing for stakeholder endorsement. You will be supported with a Business Analysts to bring all of this together.

Responsibilities

  • Analyze and organize raw platform and data warehouse data
  • Build data systems and pipelines in AWS, Delta
  • Evaluate business needs and objectives with Data Architect
  • Interpret trends and patterns
  • Conduct complex data analysis and report on results
  • Assist in the data lineage tasks
  • Assist in the empirical data analysis of source and data warehouse data sets
  • Build prototypes
  • Combine raw information from different sources
  • Explore ways to enhance data quality and reliability using AWS tooling
  • Identify opportunities for data acquisition
  • Develop analytical tools and programs
  • Collaborate with stakeholders and architects
  • Requirements analysis with BA.

Requirements and skills

  • Previous experience as a data engineer or in a similar role
  • Technical expertise with data models, data mining, and segmentation techniques
  • Knowledge of programming languages (eg Java and Python, Scala, R, SQL)
  • Hands-on experience with SQL database design
  • Great numerical and analytical skills
  • Degree in Computer Science, IT, or similar numeric field
  • Data engineering certification (eg IBM Certified Data Engineer) is a plus

Reference: 2743280310

CareerAddict

Alert me to jobs like this:

Amplify your job search:

CV/résumé help

Increase interview chances with our downloads and specialist services.

CV Help

Expert career advice

Increase interview chances with our downloads and specialist services.

Visit Blog

Job compatibility

Increase interview chances with our downloads and specialist services.

Start Test

Similar Jobs

Projects Team Manager - Pensions/GMP

Bristol, Somerset, United Kingdom

Technical Project Manager

Bristol, Somerset, United Kingdom

Principal Systems Engineer

Bristol, Somerset, United Kingdom