Be at the heart of actionFly remote-controlled drones into enemy territory to gather vital information.

Apply Now

Data Engineer

UK Tote Group
Manchester
1 week ago
Create job alert
Overview

At UK Tote Group, we're on a mission to reimagine the future of pool betting - building a modern, data-driven betting experience for millions of racing fans. Our technology powers real-time insights, supports responsible gaming, and helps us deliver trusted, customer-first products across the UK and international markets. As a Data Engineer, you'll play a key role in designing, building, and optimising the Databricks-based Lakehouse that drives real-time data, analytics, and reporting across the Tote.

What you'll be doing

You'll build streaming and batch data pipelines on AWS using Apache Spark Structured Streaming, Delta Live Tables (DLT), and Kafka (MSK), ensuring our business teams across Liquidity, Product, Marketing, Finance, and Compliance have fast, trusted data at their fingertips. This is a hands-on engineering role where you'll collaborate across engineering, BI, and product teams to deliver scalable, secure, and governed data solutions under Unity Catalogue.

Responsibilities:

  • Design, build, and optimise data pipelines using Databricks, Spark Structured Streaming, and Delta Live Tables to ensure data flows efficiently and reliably across the organisation.
  • Develop robust Bronze, Silver, and Gold Delta tables using the Medallion Architecture that support analytics, APIs, and decision-making tools.
  • Ingest data from Kafka (MSK), AWS S3, and external APIs, ensuring seamless ingestion into the Lakehouse.
  • Collaborate with BI teams to enable high-performance Power BI dashboards through Databricks SQL Warehouses, making data accessible and actionable.
  • Govern, discover, and secure data under Unity Catalog; contribute to CI/CD pipelines for Databricks jobs, notebooks, and DLT workflows.
  • Monitor, tune, and troubleshoot pipeline performance using Databricks metrics, CloudWatch, and AWS Cost Explorer.
  • Document data models, schemas, and lineage; maintain understanding of data flows and dependencies.
  • Work with Compliance and Technology to ensure the platform remains compliant with GDPR and Gambling Commission regulations.
  • Champion best practices in data platform design, observability, and cost management to shape the Tote's data ecosystem.
What we are looking for

We're looking for an experienced Data Engineer with proven expertise in building pipelines in Databricks and a strong grasp of Apache Spark, whether in PySpark or Scala, including Structured Streaming. You should have experience with Kafka (MSK) and real-time data ingestion, as well as a deep understanding of Delta Lake, Delta Live Tables, and the Medallion Architecture. A strong AWS background is important, particularly with services such as S3, Glue, Lambda, Batch, and IAM.

You'll need to be proficient in Python and SQL for data engineering and analytics, and comfortable implementing CI/CD pipelines using tools such as GitHub Actions, Azure DevOps, or Jenkins. Solid experience with Git, version control, and Spark performance tuning will help you succeed in this role. Most importantly, you'll bring a collaborative, proactive attitude and an ability to balance platform reliability with the pace of delivery.

It would be an advantage if you have experience working with streaming architectures or data governance frameworks like Unity Catalog. Familiarity with Power BI, Looker, or Tableau is desirable, as is exposure to Databricks REST APIs, Airflow, or Databricks Workflows. Knowledge of infrastructure-as-code tools such as Terraform, AWS networking fundamentals, and cost management techniques using Photon and DBU monitoring will also be beneficial.

You'll be analytical, detail-oriented, and motivated by solving complex data challenges. A self-starter by nature, you take ownership of your work and enjoy designing end-to-end solutions that deliver measurable value. You're a great communicator who can explain data concepts clearly to both technical and non-technical colleagues, and you have a passion for automation, efficiency, and data quality. Most of all, you're curious and committed to continuous learning within a fast-evolving cloud data landscape.

What's in it for you?

At the Tote you can expect a friendly working environment with a strong sense of teamwork and pride in what we do. Within this role you'll develop a broad range of skills and experiences that can enhance your career at the Tote. Additionally, our company benefits package includes;

  • Competitive Basic Salary
  • Discretionary Bonus Scheme
  • Company Shares Option Plan
  • Contributory pension scheme
  • Life insurance (4 x basic salary)
  • Simply Health Cash Plan
  • Holiday entitlement (33 days inclusive of bank holidays)
  • Study Support and opportunity for progression and development
  • Confidential 24/7 365 employee assistance helpline
  • Agile and collaborative office environment with free parking, fruit, biscuits, and drinks

Regular social events, charity events and volunteering opportunities


#J-18808-Ljbffr

Related Jobs

View all jobs

Data Engineer

Data Engineer

Data Engineer

Data Engineer

Data Engineer

Data Engineer

Subscribe to Future Tech Insights for the latest jobs & insights, direct to your inbox.

By subscribing, you agree to our privacy policy and terms of service.

Industry Insights

Discover insightful articles, industry insights, expert tips, and curated resources.

Machine Learning Recruitment Trends 2025 (UK): What Job Seekers Need To Know About Today’s Hiring Process

Summary: UK machine learning hiring has shifted from title‑led CV screens to capability‑driven assessments that emphasise shipped ML/LLM features, robust evaluation, observability, safety/governance, cost control and measurable business impact. This guide explains what’s changed, what to expect in interviews & how to prepare—especially for ML engineers, applied scientists, LLM application engineers, ML platform/MLOps engineers and AI product managers. Who this is for: ML engineers, applied ML/LLM engineers, LLM/retrieval engineers, ML platform/MLOps/SRE, data scientists transitioning to production ML, AI product managers & tech‑lead candidates targeting roles in the UK.

Why Machine Learning Careers in the UK Are Becoming More Multidisciplinary

Machine learning (ML) has moved from research labs into mainstream UK businesses. From healthcare diagnostics to fraud detection, autonomous vehicles to recommendation engines, ML underpins critical services and consumer experiences. But the skillset required of today’s machine learning professionals is no longer purely technical. Employers increasingly seek multidisciplinary expertise: not only coding, algorithms & statistics, but also knowledge of law, ethics, psychology, linguistics & design. This article explores why UK machine learning careers are becoming more multidisciplinary, how these fields intersect with ML roles, and what both job-seekers & employers need to understand to succeed in a rapidly changing landscape.

Machine Learning Team Structures Explained: Who Does What in a Modern Machine Learning Department

Machine learning is now central to many advanced data-driven products and services across the UK. Whether you work in finance, healthcare, retail, autonomous vehicles, recommendation systems, robotics, or consumer applications, there’s a need for dedicated machine learning teams that can deliver models into production, maintain them, keep them secure, efficient, fair, and aligned with business objectives. If you’re hiring for or applying to ML roles via MachineLearningJobs.co.uk, this article will help you understand what roles are typically present in a mature machine learning department, how they collaborate through project lifecycles, what skills and qualifications UK employers look for, what the career paths and salaries are, current trends and challenges, and how to build an effective ML team.