Python Developer - PySpark
Posted on Jan 8, 2020 by Applied Informatics Limited
Software architecture and development consultancy seeks suitably experienced PySpark developers as part of bid to support Office of National Statistic's (ONS) Trade and Investment Development Project.
Working as part of a small managed team in the client's Newport office, specialists will work through an established product backlog to develop and transition both new and existing system capabilities to the business as usual team responsible for publishing new and improved trade and investment statistics.
We expect this engagement to be outside IR35.
Enquiries from agencies able to offer suitable candidates welcome.
Essential skills and experience
- Strong experience of developing relational database systems using the Cloudera stack, esp. using PySpark, Hive and Impala SQL
- Able to produce technical documentation to support the development of production systems
- Good experience of developing Extract Transform and Load (ETL) pipelines
- Experience of developing continuous integration solutions, esp. using Jenkins
- Experience in version control and software review using GIT
- Proficient in the use of Jira for work item management
- Experience of working across the full development life cycle
- Awareness of good data security procedures and adherence to secure information management policies
- Ability to work collaboratively with members in other technical roles, within an agile environment, including Business Analysts and Methodologists
Highly desirable skills and experience
- Amazon Web Services (AWS)