Spark Engineer

Posted on Apr 17, 2019 by Request Technology - Anthony Honquest

Chicago, IL 60601
Information Technology
Immediate Start
$90k - $130k Annual

Spark Engineer
Chicago, IL

Prestigious Enterprise Firm is looking for a Spark Engineer for our Real Time streaming platform to enable remote monitoring of industrial assets. This is an end-to-end cloud IoT platform with ongoing field deployments. Our team is a growing hardware/software innovations group. We combine deep engineering knowledge with the best of software engineering and data science to tackle reliability and condition-monitoring applications.

Essential Functions:

Create and manage a scalable ETL pipeline usable by anyone (engineers, data scientists) for a variety of purposes (analytics, dashboards, visualizations, machine learning) including infrastructure and abstractions.

Propose services, frameworks, and other capabilities that you predict will be a necessity in the future

Build and maintain a robust data collection system that reaches across multiple sources and data marts

Monitor and tune existing infrastructure in close collaboration with the ops team

Job Requirements

Required Qualifications:

5+ years of experience in Cloud Data Infrastructure

5+ years of experience with hands on design and coding in Spark/Scala

5+ years of experience in MongoDB

3+ years of experience in data streaming using Kafka

3+ years of experience AWS

Experience in designing and scaling data infrastructure, models, and pipelines

Expertise in RDD, In-Memory computation in Spark, Fault Tolerance in Spark, Real Time stream processing

Preferred Qualifications:

Spark ML is an added advantage

Experience in Jenkins

Neo4J a plus

Skills and Abilities:

Experience deploying Cloud environments

Competence in integrating systems through APIs

Excellent technical troubleshooting skills

Ability to demonstrate excellent organization and communication skills to successfully build and maintain relationships with designers, engineers, and data scientists

Reference: 694506810

Similar Jobs

Spark/Scala Engineer

Chicago, IL

Request Technology - Kyle Honn

Data Engineer - Spark Coding

Chicago, IL

Request Technology - Robyn Honquest

Data Engineer

Chicago, IL


Data Engineer

Chicago, IL

Request Technology - Anthony Honquest