Overview
Position Title: Lead Data Engineer
Company: ITV
Location: Southampton, UK
Job Description:
Sitting in the Data Innovation function, we have an opportunity for a Data Engineer to join our data engineering team and be instrumental in delivering our data strategy. You will be part of a cross functional team and work on domain specific data products in collaboration with business stakeholders, data strategists and data scientists.
Responsibilities:
- Building highly scalable data pipelines leveraging our data platform for both batch and streaming data
- Processing large amounts of data using spark to ensure data are readily available for machine learning models, analytics and business intelligence
- Collaborating with a cross functional team including data strategist, data scientists and business stakeholders
- Optimising and maintaining existing data pipelines to improve performance, data quality and increasing business impact
- Communicating complex solutions in a clear and understandable way to both experts and non-experts
- You’ll prototype new approaches and productionise solutions at scale for our millions of active users
- Working closely with backend engineer to create source aligned data products inline with business requirements and engineering best practices
- Hire, coach and develop a team of Data Engineers
Requirements:
- Proven experience designing and implementing data pipelines leveraging Spark, Kafka, SQS
- Strong experience with Spark (Scala or PySpark)
- Ability to write production-grade Python and / or Scala code
- Experience standardising ETL pipelines
- Experience deploying code using Jenkins as CI/CD
- Good knowledge of AWS data services
About ITV
It’s an exciting time to join ITV. With the largest commercial family of channels in the UK and with our global footprint growing, we’re about to take not only our business, but also broadcasting, as we know it, to a whole new level.
