This Job Vacancy has Expired!

Senior Big Data Engineer - Kafka/Python/AZURE

Posted on Jun 23, 2020 by Orcan Intelligence

Surrey, United Kingdom
Immediate Start
£75k - £85k Annual

Big Data Solution engineer - Kafka/Python/AZURE

Our blue chip client is currently looking for Big Data Solution Engineer who is as passionate about their vision.

This includes the following responsibilities:


- Develop testing procedures, run tests and approvals, ensure quality and integrate the system into the future environment

- Maintain and optimize the relevant system, ensure performance and availability and provide release- and patch management

- Rollout the Data Platform across the international client and support the adaption of the technical services and the data capabilities

- Design the solutions of the complete use-case pipelines, from data ingestion to data processing, software deployment and feedback loop

- Understand and explain advantages and disadvantages of the proposed solutions to internal and external stakeholders

- Build and maintain data driven applications on top of data platform

- Define and apply best practices based on our platform

- Contribute to improve the data platform stack and define the final product

- Keep up with trends and evolving technology in the big data and analytics world

- Look for opportunities to improve performance, reliability, and automation

- Write technical documentation, announcements, and blog posts


- University degree in Computer Science or similar

- Proficiency in at least one programming language (we have components written in Angular, Coljure, Python, Scala, Elm etc.)

- Software engineering (micro services, design patterns, version control, automated testing, concurrent programming etc.)

- Continuous integration, deployment, and delivery

- Competence in running big data workloads in production at scale

- Data Engineering Pattern (Ingestion, Transformation, Storage, Consumption etc.)

- Event-based systems (deep knowledge on the Kafka confluent stack)

- Databases (eg PostgreSQL, Neo4J, Stardog)

- Cloud Storage (Azure Datalake or AWS S3 with EMR)

- Distributed systems (eg Spark)

- Knowledge Graph (eg Stardog)

- General understanding of Infrastructure, Orchestration and IT Security Principles (especially on an enterprise level)

- Advanced Experience with Linux

- Containerization/Container Orchestration (Docker & Kubernetes)

- DevOps mindset (you build it, you run it; taking responsibility for your work)

- Willingness and ability to learn new technologies and tools

- Team player open to work in an agile environment

- Fluent English (written and spoken) is a must, other languages are a plus

- Capability of result oriented communication with people from different departments with different skill sets

- Readiness to mentor new team members and junior colleagues

- Ability to carry out tasks independently in non-standardized (ever-changing) work situations

If you'd like to explore this opportunity further, please send your CV now for immediate consideration

Reference: 902763756

Set up alerts to get notified of new vacancies.