Big Data - Python/Scala Developer
Posted on Aug 28, 2020 by Experis AG
Big Data - Python/Scala Developer
Experis is the global leader in professional resourcing and project-based workforce solutions. Our suite of services range from interim and permanent recruitment to managed services and consulting, enabling businesses to achieve their goals. We accelerate organisational growth by attracting, assessing and placing specialised professional talent.
On behalf of our client, a leading financial institution in Zürich, we are looking for a Python/Scala Developer with extensive Big Data experience.
Overview of the project:
- The successful candidate will work in the team responsible for development of Global Trading Services Data Platform. The project aims to build a scalable platform for processing streaming end-of-day data, enriching it further, extracting business metrics and presenting the data to the users using off-the-shelf dashboard solutions.
- The successful candidate should be capable to pick up all tasks related to the entire life cycle of product development, ie gathering requirements and analysis, design of software architecture, implementation, design and write different types of tests, packaging, deployment, support and maintenance software in production; know-how and understand Agile methodology and its tools.
- Development of software solutions in Business Intelligence/data analytics/data science space
- Contribute to modernization of the software solution, drive adoption of newer technologies with a focus on scalability
- Automated build, packaging and deployment process for production deployments.
- Develop software components responsible for sourcing intraday and end-of-day data, processing this data further and extracting analytical insights from this data
- Develop monitoring capabilities for the data processing pipelines in Production
- Actively participate in the design and technology review of the software components developed in the team
- Evolve overall architecture of the solution with the use of latest technologies available in the bank
- Work to streamline development process and to improve software performance
- Contribute to integration testing (automated and manual) efforts as required
- Collaborate with platform management and other team members on the requirements, preparing the releases and delivering the applications to production
- Assist to resolve incidents involving Production system (3rd level support)
- At least 3 years of experience developing data-intensive software solutions using Python and Scala
- Extensive experience with Big Data technologies, such as Hadoop, YARN, Spark, Impala, Kudu, NiFi, Oozie
- Practical experience with major Hadoop distributions (Cloudera, Hortonworks, Databricks); good knowledge of individual components; ability to troubleshoot issues in Production
- Experience in building data streaming pipelines using Apache Spark or Apache Flink (ideally both)
- Experience with Datamodelling and SQL query language
- Good understanding of security concepts
- Experience in successfully integrating emerging open-source technologies and ability to shape the solution architecture
- Experience with Elastic technology stack (Elastic/Kibana)
- Practical experience with Apache Druid/Apache Ignite
- Experience with automated provisioning and configuration of open-source distributed systems
- Exposure to Kubernetes
Interested in this opportunity? Kindly send us your CV today through the link in the advert. However should you have any questions please contact Sophia Biafora.