Big data Cloud Architect
Posted on Jun 1, 2021 by Cyberteam
Big data Cloud Architect
Big data Cloud Architect urgently needed for a long term opportunity based in Belgium, Leuven location
What does this function mean?
The customers group is currently creating a new big data platform in AWS. On top of the platform, we are building a platform that focuses on supporting data science and other technical computing workflows. You will participate in this effort, as a valuable team player, to analyse, design, and implement the platform in close collaboration with data scientists.
What do we expect from you?
As a platform engineer for data science, you are responsible for engineering a well architected data science platform with particular emphasis in interactive computing and highly automated model life cycle management based on MLOps. Main considerations for the design include security, maintainability, flexibility, user-friendliness and integration between components on the platform.
You work closely with data scientists to understand their functional needs and translate them in an actual solution. Then, you rapidly iterate over the solution based on frequent feedback from data scientists.
Due to how the platform is used in the group, you are also responsible for documenting guidelines for future platform operators.
An ideal candidate should have:
Decent Cloud experience (doesn t matter which Cloud, as long as it is one of the big three (GCP, Azure or AWS) and Infrastructure-as-Code experience is of course equally valuable.
- Good knowledge of user management (authentication and authorization) with AWS products and OSS solutions
- Good grasp of big data ecosystem and tooling especially Apache Spark and related technologies
- Good verbal and written communication skill in English
- Willingness to work in close collaboration with data scientists and other engineers from different teams
- Basic understanding of AWS products related to machine learning (AWS SageMaker)
- Prior experience with Infrastructure-as-Code (IAC) tools such as Terraform, Pulumi, AWS CloudFormation etc
- An ability to work in a highly dynamic environment
- A can-do attitude to cope with non-technical constraints
In time you will develop in-depth and practical experience in:
- AWS SageMaker
- Other AWS services such as Cognito, Lambda, CloudWatch, ECR, Athena etc
- AWS S3
- Apache Spark
Shortlisting immediately interviews starting 02/06/21