Data Engineer - GCP
Posted on Jul 8, 2021 by Appsbroker Ltd
Hiring at the Intermediate and Senior levels
Are we looking for you?
As a Data professional, you have ambitions to work on transformational projects, building big data solutions and designing large-scale data processing systems, growing as a true consultant. You'll enjoy partnering with enterprise level customers on how to ingest, store, process, analyse, explore and visualise data on Public Cloud platforms.
Complex business problems excite you, and you want to use Big Data, Auto-ML, ML and visualisation tech (with others) to solve them.
In summary, here you will be designing cloud enterprise solutions architecture and helping to build big data solutions for customers by integrating structured, semi-structured and unstructured data on low latency platforms.
Are you looking for us?
Appsbroker is Google's largest Premier Partner in Europe and a MSP.
We deliver leading-edge cloud services to some of the world's most exciting brands, in sectors from retail through automotive to finance.
We expose you to some of the most exciting Cloud transformation and optimisation project opportunities utilising Google Cloud (GCP).
Full training and certification on GCP are provided as part of an extensive on-boarding program even if your background is more AWS/Azure however working knowledge of GCP is required.
Example Data Project - Multinational Investment Bank - Largest in Europe
Increasing runtimes and lack of scalability of on-prem Risk Reporting Platform batch ETL process led to challenges meeting increasing volume of regulatory reporting requirements. Lack of dedicated development and UAT led to resource contention and hampered the agility of development and deployment of fixes.
What your future team delivered
Deployed a hybrid solution leveraging the performance and scalability of Cloud Storage, Compute Engine Managed Instance Groups and BigQuery to move the bulk of the data processing to GCP. All data is protected at rest with Customer Managed Encryption Keys (CMEK) and sensitive or PII data is encrypted prior to transfer to GCP and decrypted on premise once data processing is complete. Result was 18x faster data processing on GCP, leading security and cost savings for the bank.
Day to Day
Appsbroker is dynamic and the role is varied (there is no 'typical day') so you'll get involved in a wide range of activities (more production than POC) but here is what to expect:
- Identify, scope and participate in the design and delivery of cloud data platform solutions, data storage, movement and orchestration solutions utilising GCP
- Design and execute a platform modernisation approach for customers' data environments
- Design, coordinate and execute pilots, prototypes or proof of concepts, provide validation on specific scenarios and provide deployment guidance
- Collaborate and share best practices/insights with engineering colleagues and the architect community
- Travel to client sites as appropriate (when restrictions allow)
What skills we look for
- Google Cloud Platform (GCP) experience
- Able to consult on various technologies around databases, data warehousing and big data eg BigQuery, Oracle, Redshift, Teradata, Hadoop.
- Articulate communication skills to explain complex solutions to external or internal customers in a clear and concise manner, aligned with the ability to adjust style to a varied audience within an organisation.
- Exposure to data science modelling, analytics & BI reporting (Tableau, Looker, Periscope, DataStudio).
- History of working with Data warehouse solutions (on-premise & Public Cloud).
- Good experience of SQL on relational & non-relational database (RedShift, BigQuery, MySQL, etc).
- Analytical and design skills around the full end-to-end ETL life cycle of data pipelines in large enterprise settings
- Practical experience of data curation, cleansing, creation and maintenance of data sets for business purposes
Appsbroker is committed to building a diverse and inclusive community where we can all be ourselves and succeed on merit.