Big Data Engineer
Posted on Jul 22, 2021 by Source Technology
We are looking for a Big Data Support Engineer with 2+ years of experience in supporting and administering a Hadoop/Cloudera environment.
This is a permanent English speaking role, working for a global client.
They offer an excellent package, including relocation where required.
Roles and responsibility:
- Continuous improvement of existing big data platform (size > 5PB) hosted on Cloudera stack.
- Adding data nodes, Cloudera services and support.
- Handling hive, impala, Spark processing issues.
- Integration of API calls with Hadoop and analytical processing layer, using python and Shell Scripting.
- Setting jobs for cluster admin automation.
- Platform tenant support, such as access, setting up new control-M jobs, onboarding tenants, resolving security/access issues.
- Setting up jobs for feed transfer, ingestions.
- Knowledge of Hadoop components such as HDFS, Spark, HUE, Impala, hive, HBase and Kafka.
- Should able to understand Hadoop security such as Kerberos, impersonification, ACLs etc.
- Basic programming skills in python and Shell.
- Should know batch processing framework such as Rundeck, control M.
- Strong Unix skills will be advantage
- Hadoop admin using Cloudera will be advantage
- Excellent organizational skills as well as a very detailed and efficient work approach