Data Engineer Manager

Canary Wharf
10 months ago
Applications closed

Related Jobs

View all jobs

Data Engineer Manager

Consultant - Manager, Data Engineer, AI & Data, Defence & Security

Data Engineer (Data Science)

Data Engineer

Data Engineering Manager

Data Engineering Manager

Data Engineer Manager

Perm

London

£75,000pa - £85,000pa

Role Summary

The Data Engineer Manager is responsible drive the design, development, and optimization of data solutions in the data infrastructure. In addition to fostering the growth of a skilled team, you will play a pivotal role in delivering the data applications, infrastructure, and services, ensuring they align with organizational goals and industry best practices.

As part of the Technology Hub the Data Engineer Manager will work very closely with all teams across the business. The role is instrumental in defining and upholding a clear vision for the integrity of data life cycle management aligning the strategic goal of becoming a centre of expertise. Additionally, it ensures stewardship of business data and technical architecture, fostering innovation and reliability across all data initiatives.

Key Responsibilities



Mentor the data engineering team to design and implement complex, tailored data solutions that support processing of high volumes of data across all schemes and applications.

*

Establish and support the technical vision and strategy for a robust data architecture that aligns with the overall strategy, with a strong focus on ensuring security for all structured data.

*

Establish and maintain robust operational support and monitoring systems to ensure the reliable performance of critical systems in live environments.

*

Facilitate the adoption and implementation of continuous delivery practices while advocating for the use of cloud solutions.

*

Design, implement, and optimize end-to-end data pipelines and solutions on Azure, ensuring data quality, reliability, and security throughout. Oversee the integration of both structured and unstructured data sources.

*

Oversee project timelines, scope, and deliverables to ensure successful execution, while actively monitoring progress and addressing risks proactively.

*

Implement best practices for process improvements, cost optimization and monitoring.

*

Continuously evaluate and improve the Azure data platform to enhance performance and scalability.

*

Collaborate with stakeholders to understand business requirements and translate them into technical solutions.

*

Develop and implement a comprehensive data governance framework to ensure data quality, security, and compliance across the data applications.

*

Design, evaluate impacts, perform technical design reviews, and approve technical designs as part of the design authority process.

*

Establish and maintain comprehensive documentation for all data engineering processes, pipelines, and systems.

*

Implement best practices for maintaining version control and traceability of documentation.

*

Foster continuous learning and adoption of the latest technologies while mentoring and training the data engineering team.

Key Requirements

Essential:

*

Minimum 6 years’ experience in Data Engineering, Data Architecture & Governance frameworks.

*

Minimum 4 years' experience with Python, preferably PySpark.

*

Experience leading small teams of Engineers.

*

Excellent communication and stakeholder management abilities.

*

Strong expertise in Azure: ADLS, Databricks, Stream Analytics, SQL DW, Synapse, Databricks, Azure Functions, Serverless Architecture, ARM T emplates, DevOps.

*

Hands-on experience with ETL/ELT processes and data warehousing.

*

Solid understanding of data security and compliance standards.

*

Experience with DevOps practices and tools (e.g., CI/CD pipelines, Azure DevOps).

*

The ability to simplify complex technical issues for a non-technical stakeholder audience.

*

Capable of understanding business needs and requirements while providing valuable, insightful recommendations.

*

Skilled in delivering presentations and technical reports clearly and persuasively

Subscribe to Future Tech Insights for the latest jobs & insights, direct to your inbox.

By subscribing, you agree to our privacy policy and terms of service.

Industry Insights

Discover insightful articles, industry insights, expert tips, and curated resources.

Jobs

How to Write a Machine Learning Job Ad That Attracts the Right People

Machine learning now sits at the heart of many UK organisations, powering everything from recommendation engines and fraud detection to forecasting, automation and decision support. As adoption grows, so does demand for skilled machine learning professionals. Yet many employers struggle to attract the right candidates. Machine learning job adverts often generate high volumes of applications, but few applicants have the blend of modelling skill, engineering awareness and real-world experience the role actually requires. Meanwhile, strong machine learning engineers and scientists quietly avoid adverts that feel vague, inflated or confused. In most cases, the issue is not the talent market — it is the job advert itself. Machine learning professionals are analytical, technically rigorous and highly selective. A poorly written job ad signals unclear expectations and low ML maturity. A well-written one signals credibility, focus and a serious approach to applied machine learning. This guide explains how to write a machine learning job ad that attracts the right people, improves applicant quality and strengthens your employer brand.

Education Jobs

Maths for Machine Learning Jobs: The Only Topics You Actually Need (& How to Learn Them)

Machine learning job adverts in the UK love vague phrases like “strong maths” or “solid fundamentals”. That can make the whole field feel gatekept especially if you are a career changer or a student who has not touched maths since A level. Here is the practical truth. For most roles on MachineLearningJobs.co.uk such as Machine Learning Engineer, Applied Scientist, Data Scientist, NLP Engineer, Computer Vision Engineer or MLOps Engineer with modelling responsibilities the maths you actually use is concentrated in four areas: Linear algebra essentials (vectors, matrices, projections, PCA intuition) Probability & statistics (uncertainty, metrics, sampling, base rates) Calculus essentials (derivatives, chain rule, gradients, backprop intuition) Basic optimisation (loss functions, gradient descent, regularisation, tuning) If you can do those four things well you can build models, debug training, evaluate properly, explain trade-offs & sound credible in interviews. This guide gives you a clear scope plus a six-week learning plan, portfolio projects & resources so you can learn with momentum rather than drowning in theory.

Jobs

Neurodiversity in Machine Learning Careers: Turning Different Thinking into a Superpower

Machine learning is about more than just models & metrics. It’s about spotting patterns others miss, asking better questions, challenging assumptions & building systems that work reliably in the real world. That makes it a natural home for many neurodivergent people. If you live with ADHD, autism or dyslexia, you may have been told your brain is “too distracted”, “too literal” or “too disorganised” for a technical career. In reality, many of the traits that can make school or traditional offices hard are exactly the traits that make for excellent ML engineers, applied scientists & MLOps specialists. This guide is written for neurodivergent ML job seekers in the UK. We’ll explore: What neurodiversity means in a machine learning context How ADHD, autism & dyslexia strengths map to ML roles Practical workplace adjustments you can ask for under UK law How to talk about neurodivergence in applications & interviews By the end, you’ll have a clearer sense of where you might thrive in ML – & how to turn “different thinking” into a genuine career advantage.

🍪 We use cookies to enhance your browsing experience on our website. By continuing to use this site, you consent to the use of cookies as described in our Cookie Policy. You can review our Cookie Policy for more information.