Be at the heart of actionFly remote-controlled drones into enemy territory to gather vital information.

Apply Now

Senior Data Engineer (contract)

Methods Business and Digital Technology Ltd
Ledbury
3 days ago
Create job alert
Overview

Senior Data Engineer — On-site — Full time

Methods Business and Digital Technology Limited is a UK-based IT Services Consultancy with a strong focus on transforming public sector operations and data solutions for central government departments and agencies.

Methods was acquired by the Alten Group in early 2022.

Responsibilities

We are seeking a seasoned Senior Data Engineer to design, build, and maintain sophisticated data infrastructure systems that operate across both on-premises and Azure cloud environments. The role involves deploying and managing scalable data operations that support advanced analytics and data-driven decision-making, contributing to organisational growth and innovation.

  • Develop and Manage Data Pipelines: design, construct, and maintain efficient and reliable data pipelines using Python/Go/Azure Data Factory, supporting streaming and batch processing across structured, semi-structured, and unstructured data in on-premises and Azure environments.
  • Hybrid Cloud and Data Storage Solutions: implement and manage data storage solutions leveraging on-premises infrastructure and Azure, ensuring seamless data integration and accessibility across platforms.
  • Containerisation and Orchestration: utilise Docker for containerisation and Kubernetes for orchestration to enable scalable deployment across cloud and on-premises environments.
  • Workflow Automation: use tools such as Azure Data Factory to automate data flows and manage complex workflows within hybrid environments.
  • Event Streaming Experience: utilise event-driven technologies such as Kafka and NATS to handle real-time data streams.
  • Security and Compliance: manage security setups and access controls, including tools like Keycloak to protect data integrity and comply with legal standards.
  • Database Development: design and develop PostgreSQL databases with high performance and availability across deployment scenarios.
Essential Skills and Experience
  • Strong Python Skills: expertise in Python for scripting and automating data processes across varied environments.
  • Experience with ETL/ELT: demonstrable experience in developing and optimizing ETL or ELT workflows in hybrid (on-premises and Azure) environments.
  • Expertise in Hybrid Cloud Data Architecture: knowledge of integrating on-premises infrastructure with Azure cloud services.
  • Containerisation and Orchestration Expertise: solid experience with Docker, GitHub and Kubernetes in managing applications across on-premises and cloud platforms.
  • Proficiency in Workflow Automation Tools: practical experience with Azure Data Factory in relevant environments.
  • Experience in Event Streaming: proven ability in managing and deploying event streaming platforms like Kafka and NATS.
  • Data Security Knowledge: experience with implementing security practices and tools, including Keycloak, across multiple platforms.
  • Search and Database Development Skills: strong background in Elasticsearch and PostgreSQL across on-premises and cloud infrastructures.
Your Impact

You will empower business leaders to make informed decisions by delivering timely, accurate, and actionable data insights from a robust, hybrid infrastructure. You will champion adoption of modern data architectures, mentor team members, and advance engineering practices to cultivate a data-driven culture within the organisation.

Desirable Skills and Experience
  • Azure and Other Relevant Certifications: cloud and on-premises certifications are beneficial.
  • Data Engineering Experience: minimum of 5 years in data engineering with significant exposure to on-premises and cloud infrastructure.
  • DevOps Experience: some DevOps engineering experience would be preferable.


#J-18808-Ljbffr

Related Jobs

View all jobs

Senior Data Engineer

Senior Data Engineer

Senior Data Engineer

Senior Data Engineer

Senior Data Engineer | Cambridge | Greenfield Project

Senior Data Engineer - 12 month FTC

Subscribe to Future Tech Insights for the latest jobs & insights, direct to your inbox.

By subscribing, you agree to our privacy policy and terms of service.

Industry Insights

Discover insightful articles, industry insights, expert tips, and curated resources.

Why Machine Learning Careers in the UK Are Becoming More Multidisciplinary

Machine learning (ML) has moved from research labs into mainstream UK businesses. From healthcare diagnostics to fraud detection, autonomous vehicles to recommendation engines, ML underpins critical services and consumer experiences. But the skillset required of today’s machine learning professionals is no longer purely technical. Employers increasingly seek multidisciplinary expertise: not only coding, algorithms & statistics, but also knowledge of law, ethics, psychology, linguistics & design. This article explores why UK machine learning careers are becoming more multidisciplinary, how these fields intersect with ML roles, and what both job-seekers & employers need to understand to succeed in a rapidly changing landscape.

Machine Learning Team Structures Explained: Who Does What in a Modern Machine Learning Department

Machine learning is now central to many advanced data-driven products and services across the UK. Whether you work in finance, healthcare, retail, autonomous vehicles, recommendation systems, robotics, or consumer applications, there’s a need for dedicated machine learning teams that can deliver models into production, maintain them, keep them secure, efficient, fair, and aligned with business objectives. If you’re hiring for or applying to ML roles via MachineLearningJobs.co.uk, this article will help you understand what roles are typically present in a mature machine learning department, how they collaborate through project lifecycles, what skills and qualifications UK employers look for, what the career paths and salaries are, current trends and challenges, and how to build an effective ML team.

Why the UK Could Be the World’s Next Machine Learning Jobs Hub

Machine learning (ML) is becoming essential to industries across the globe—from finance and healthcare to retail, logistics, defence, and the public sector. Its ability to uncover patterns in data, make predictions, drive automation, and increase operational efficiency has made it one of the most in-demand skill sets in the technology world. In the UK, machine learning roles—from engineers to researchers, product managers to analysts—are increasingly central to innovation. Universities are expanding ML programmes, enterprises are scaling ML deployments, and startups are offering applied ML solutions. All signs point toward a surging need for professionals skilled in modelling, algorithms, data pipelines, and AI systems. This article explores why the United Kingdom is exceptionally well positioned to become a global machine learning jobs hub. It examines the current landscape, strengths, career paths, sector-specific demand, challenges, and what must happen for this vision to become reality.