This Job Vacancy has Expired!

Lead Data Engineer

Tank Recruitment Limited

Posted on Jan 4, 2023 by Tank Recruitment Limited

London, United Kingdom
IT
Immediate Start
Annual Salary
Full-Time

Data Engineer Lead

Perm - Fully remote

On behalf of a key client we are resourcing for a fully remote Lead data engineer, this is a permanent fully remote role

Role summary

The Data Engineering Lead will be responsible for the design, development and maintenance of robust scalable and high-performance data platform with Snowflake and Fivetran including ETL/ELT

Responsibilities

  • Demonstrate experience architecting and implementing solutions in cloud environments under an agile framework. Being comfortable defining and maintaining a high standard for development in close collaboration with data analysts, data scientists, product owners, DBAs and external stakeholders of varied seniority. Use your skills as a critical thinker who is able to anticipate and prevent challenges before they occur with solid design and a careful attention to detail and will be commercial enough to understand the purpose of your output - being able to competently challenge assumptions, business cases and approaches if there could be a better option.
  • Responsible for the analysis, design, development, testing and maintenance of ELT solutions, predictive models, tabular models and data marts.
  • Ownership for cloud-based data lake, using serverless architectures wherever possible to absolutely minimise the need for platform maintenance and permit exponential growth with diminishing effort.
  • Delivery of data governance which informs a single source of truth in respect to information.
  • To highly automate the entire SDLC according to best practice patterns and practices which result in a high velocity data engineering service to the business.
  • To work according to best practices and patterns which enable us to move at the highest velocity in managing change.
  • Helping users adhere to agile practices utilising a toolset that includes Clickup, AzureDevOps and continuous integration build systems.
  • Establish and maintain a best-in-class approach to testing, leaning heavily towards full automation of unit test, regression test and integration test and leading by example in respect to testing and data quality processes.
  • Own and continuously improve development standards for the data engineering team. This includes coding standards, version control practices, peer reviews, and the creation and versioning of ETL documentation artefacts.
  • Perform root cause analysis for troubleshooting production run-time issues.
  • To rigorously investigate failed releases, identifying, prioritising and eliminating root causes as a process for continuous improvement.
  • Work with business SMEs and Business Analysts to convert business needs into technical requirements.
  • Ownership of a comprehensive data catalog for all metadata, business rules and logic.
  • Optimize SQL, Python, R code performance and processing methods.
  • Perform design validation, reconciliation and error handling across the entire ELT process (initial, incremental).

Required Skills

  • Strong programming knowledge and skills with Snowflake
  • Strong programming using Python and R.
  • Solid experience working in an environment where analytics for data surfaces in PowerBI and Excel.
  • Strong background Microsoft SQL 2019- our application database which is ingested into Snowflake.
  • Demonstrated skill and ability in the development of data warehouse using best practices, including star and snowflake schemas, FACT tables, cubes and tabular models and associated languages such as TSQL.
  • Good understanding of OLTP and OLAP databases
  • Demonstrably excellent understanding of Kimball Data Warehouse concepts in theory and practice
  • Excellent on building and tuning queries in TSQL
  • Evidence of heavy involvement in creation of data warehouse from inception to delivery.
  • Experience with Agile development practices
  • Experience with continuous integration and delivery in theory and practice

Desirable skills

  • Databricks, Apache Spark, Hadoop and Hive or another similar platform utilising map-reduce functionality such as EMR.
  • Working with Supply chain
  • A genuine interest in sustainability and a desire to save the planet using technology.
  • Presenting data engineering status/projects to senior management

Reference: 1844038902

CareerAddict

Alert me to jobs like this:

Digital marketing manager in London, Full-time

Amplify your job search:

CV/résumé help

Increase interview chances with our downloads and specialist services.

CV Help

Expert career advice

Increase interview chances with our downloads and specialist services.

Visit Blog

Job compatibility

Increase interview chances with our downloads and specialist services.

Start Test

Similar Jobs

Lead Data Engineer

Manchester, Lancashire, United Kingdom

Lead Data Engineer

City of London, City and County of the City of London, United Kingdom

Lead Data Engineer

Preston, Lancashire, United Kingdom

Lead Data Engineer

Framwellgate Moor, County Durham, United Kingdom

Lead Data Engineer

London, United Kingdom

Lead Data Engineer

Canary Wharf, Greater London, United Kingdom

Lead Data Engineer

Brighton, East Sussex, United Kingdom

Lead Data Engineer - Azure

Newcastle upon Tyne, Tyne & Wear, United Kingdom

Leanix Architect - Remote

Not Specified, United Kingdom