Big Data Platform Engineer
Posted on Oct 16, 2019 by Ecolab
As the Big Data Platform Engineer will be a key member of the Data and Analytics team leading and coordinating the development around Data and Analytics initiatives. This role will enable the development and building of the big data solutions that our big data solutions architects have designed. You will be the Lead technical Big Data Engineer
What you will do
- Manage the technical data engineering of global, cross-divisional Data and Analytics projects based on a multi-year roadmap.
- Design big data engineering solutions using your experience with Hadoop-based technologies
- Design and build (with a team of engineers) large-scale data processing systems
- Coordinate the technical development and deployment activities across multiple Data and Analytics projects.
- Enhance/develop big data engineering standards and guiding principles for the platform to be followed by our internal and external data engineers
- Work with our big data architects to design highly scalable distributed systems using our open-source tools
- Achieve technical excellence by advocating for and adhering to lean-agile engineering principles and practices such as API-first design, simple design, continuous integration, version control, and automated testing
- Identify key dependencies across multiple projects and enable the development of common objects for use across multiple analytics implementations.
- Work with delivery teams from concept to deployment
- Develop prototypes and proof of concepts for the selected solutions with our big data architects
- 10+ years working in an IT environment
- 5+ years' experience as a big data engineer on Apache-platforms (Hortonworks, Cloudera, )
- 3+ years' experience with tools in the Big Data/Hadoop ecosystem (Hive, Nifi, Kafka, Ambari, Spark, Ranger, and/or Atlas)
- 3+ years' experience as the lead big data engineers on multiple Data and Analytics projects
- 2+ years' experience as the technical lead working with multiple engineers.
- Experience identifying common objects across multiple initiatives and leading consensus around building shareable objects
- Experience in delivering solutions on large Data and Analytics projects
- Experience in building data pipelines w/Kafka and Nifi
- Experience implementing complex big data projects with a focus on collecting, parsing, managing, analyzing and visualizing large sets of data to turn information into insights using multiple platforms.
- Experience with hardening (putting into production) data models developed by our data scientists
- Bachelor's degree in a technical area or equivalent.
- Experience working in a multi-technology environments a plus.
- Ability to decide on the needed hardware and software design needs and act according to the decisions.
- Experience ingesting data from SAP platforms is a plus.
- Demonstrable experience with Data Analysis and/or Data Modeling
- Advanced knowledge of data tools, techniques and manipulation
- Excellent communication skills, particularly those relating to complex findings and presenting them to ensure audience appeal at various levels of the organization
- Ability to integrate research and best practices into problem avoidance and continuous improvement
- Strong analytical and problem-solving skills
- Strong interpersonal skills to successfully team with software vendors, outsourcing partners and internal business and IT associates.
- Must be able to perform as an effective member in a team-oriented environment, maintain a positive attitude, and achieve desired results while working with minimal supervision
Our Commitment to Diversity and Inclusion
At Ecolab, we believe the best teams are diverse and inclusive, and we are on a journey to create a workplace where every associate can grow and achieve their best. We are committed to fair and equal treatment of associates and applicants. We recruit, hire, promote, transfer and provide opportunities for advancement on the basis of individual qualifications and job performance. In all matters affecting employment, compensation, benefits, working conditions, and opportunities for advancement, we will not discriminate against any associate or applicant for employment because of race, religion, color, creed, national origin, citizenship status, sex, sexual orientation, gender identity and expressions, genetic information, marital status, age, disability, or status as a covered veteran.
In addition, we are committed to furthering the principles of Equal Employment Opportunity (EEO) through Affirmative Action (AA). Our goal is to fully utilize minority, female, disabled and covered veteran individuals at all levels of the workforce. Ecolab is a place where you can grow your career, own your future and impact what matters.