Data Engineering Manager - Big Data

AWS Data Engineering Manager (1 Braham Street, London, United Kingdom) Recruiter:  BT Group was the world’s first telco and our heritage in the sector is unrivalled. As home to several of the UK’s most recognised and cherished brands – BT, EE, Openreach and Plusnet, we have always played a critical role in creating the future, and we have reached an inflection point in the transformation of our business. 

 

Over the next two years, we will complete the UK’s largest and most successful digital infrastructure project – connecting more than 25 million premises to full fibre broadband. While we are through the most capital-intensive phase of our fibre investment, meaning we can reward our shareholders for their commitment and patience, we are absolutely focused on how we organise ourselves in the best way to serve our customers in the years to come. Together with our application of AI and technology, we are on a path to creating the UK’s best telco, reimagining the customer experience and relationship with one of this country’s biggest infrastructure companies.

 

The AWS Data Engineering Manager plays a mission-critical role in enabling BT’s DD Ops transformation by leading the design, implementation, and optimisation of data infrastructure that powers intelligent, automated operations. This role manages a team of skilled data engineers and is responsible for building and maintaining scalable, secure, and high-performance data pipelines that capture, manage, store, and utilise both structured and unstructured data from internal and external sources. These pipelines are foundational to delivering real-time insights, predictive analytics, and AI Ops capabilities across BT’s mobile and fixed networks.

 

Team Leadership & Coaching: Lead and mentor a team of data engineers, guiding them through complex, open-ended projects and fostering a high-performance, collaborative culture. Shape the technical vision of the data engineering function, contributing deep expertise across big data, systems design, machine learning, and cloud infrastructure.

Data Infrastructure & Optimisation: Oversee the development and maintenance of accurate, high-quality datasets and optimised codebases that support data products, pipelines, and scalable architectures.

Agile Delivery & Best Practices: Ensure teams follow Agile methodologies and engineering best practices to consistently deliver high-quality, production-ready solutions. Data Quality & Visibility: Coordinate the use of internal and external data sources to define and monitor key indicators of data quality, pipeline health, and infrastructure performance.

Resource Management: Allocate engineering resources effectively to address priority issues, ensuring timely responses and measurable outcomes.

Translate business objectives into scalable, end-to-end data solutions that meet customer needs and align with strategic timelines.

Manage relationships with outsourced partners and suppliers, setting clear expectations around deliverables, quality, timelines, and cost.

Champion emerging trends in data engineering, continuously developing and sharing knowledge to drive innovation and technical excellence.

Talent Development: Coach and develop team members through upskilling, performance management, and recruitment to build future-ready capabilities.

Process Improvement : Lead initiatives to enhance data engineering workflows, tools, and practices for greater efficiency and impact.

 

Leadership in data engineering and Agile delivery

Advanced knowledge of AWS data services (e.g. Expertise in big data technologies and distributed systems

Strong coding and optimisation skills (e.g. Python, Spark, SQL)

Data quality management and observability

Proven experience managing data engineering teams in cloud-native environments

Track record of delivering scalable data solutions aligned with business goals

Hands-on experience with modern data platforms, pipelines, and governance frameworks

Familiarity with machine learning workflows and data science enablement

Although these roles are listed as full-time, if you’re a job share partnership, work reduced hours, or any other way of working flexibly, please still get in touch.

Studies have shown that women and people who are disabled, LGBTQ+, neurodiverse or from ethnic minority backgrounds are less likely to apply for jobs unless they meet every single qualification and criteria. Were committed to building a diverse, inclusive, and authentic workplace where everyone can be their best, so if youre excited about this role but your past experience doesnt align perfectly with every requirement on the Job Description, please apply anyway - you may just be the right candidate for this or other roles in our wider team.