Be at the heart of actionFly remote-controlled drones into enemy territory to gather vital information.

Apply Now

Data Engineer

Data Engineering
Wigan
4 days ago
Create job alert
Overview

At UK Tote Group, we’re reimagining the future of pool betting with a data‑driven, modern experience. Our technology powers real‑time insights, supports responsible gaming, and delivers trusted, customer‑first products across the UK and international markets. As a Data Engineer, you will design, build, and optimise the Databricks‑based Lakehouse that drives real‑time data, analytics, and reporting across the Tote. You’ll build streaming and batch data pipelines on AWS using Apache Spark Structured Streaming, Delta Live Tables (DLT), and Kafka (MSK), ensuring business teams across Liquidity, Product, Marketing, Finance, and Compliance have fast, trusted data at their fingertips. This is a hands‑on engineering role collaborating with engineering, BI, and product teams to deliver scalable, secure, and governed data solutions under Unity Catalogue.


What you’ll be doing

You’ll design, build, and optimise data pipelines using Databricks, Spark Structured Streaming, and Delta Live Tables to ensure data flows efficiently and reliably across the organisation. By applying the Medallion Architecture, you’ll develop Bronze, Silver, and Gold Delta tables that serve as the foundation for analytics, APIs, and decision‑making tools. You’ll integrate data from Kafka (MSK), AWS S3, and external APIs, ensuring seamless ingestion into the Lakehouse. You’ll work closely with BI teams to enable high‑performance Power BI dashboards through Databricks SQL Warehouses, making data more accessible and actionable across the business. You’ll ensure all data is governed, discoverable, and secure under Unity Catalogue, and you’ll contribute to continuous delivery by implementing and maintaining CI/CD pipelines for Databricks jobs, notebooks, and DLT workflows. Monitoring, tuning, and troubleshooting pipeline performance will also be part of your day‑to‑day, using Databricks metrics, CloudWatch, and AWS Cost Explorer to ensure optimal performance and efficiency. You’ll document data models, schemas, and lineage, maintaining a clear understanding of data flows and dependencies. You’ll help ensure the platform remains fully compliant with GDPR and Gambling Commission regulations alongside Compliance and Technology colleagues. Finally, you’ll champion best practices in data platform design, observability, and cost management, helping shape the Tote’s data ecosystem.


Qualifications

  • Experience building pipelines in Databricks with strong knowledge of Apache Spark (PySpark or Scala), including Structured Streaming.
  • Experience with Kafka (MSK) and real‑time data ingestion; deep knowledge of Delta Lake, Delta Live Tables, and the Medallion Architecture.
  • Strong AWS background (S3, Glue, Lambda, Batch, IAM).
  • Proficient in Python and SQL for data engineering and analytics; able to implement CI/CD pipelines using GitHub Actions, Azure DevOps, or Jenkins.
  • Solid experience with Git, version control, and Spark performance tuning.
  • Collaborative, proactive attitude with focus on reliability and timely delivery.
  • Nice‑to‑have: experience with streaming architectures or data governance frameworks like Unity Catalog; familiarity with Power BI, Looker, or Tableau; exposure to Databricks REST APIs, Airflow, or Databricks Workflows; knowledge of Terraform, AWS networking, and cost management (Photon and DBU monitoring).
  • Strong communication skills and a drive for automation, data quality, and continuous learning.

Benefits

  • Competitive Basic Salary
  • Discretionary Bonus Scheme
  • Company Shares Option Plan
  • Contributory pension scheme
  • Life insurance (4 x basic salary)
  • Simply Health Cash Plan
  • Holiday entitlement (33 days inclusive of bank holidays)
  • Study Support and opportunity for progression and development
  • Confidential 24/7 365 employee assistance helpline
  • Agile and collaborative office environment with free parking, fruit, biscuits, and drinks
  • Regular social events, charity events and volunteering opportunities


#J-18808-Ljbffr

Related Jobs

View all jobs

Data Engineer

Data Engineer

Data Engineer

Data Engineer

Data Engineer

Data Engineer

Subscribe to Future Tech Insights for the latest jobs & insights, direct to your inbox.

By subscribing, you agree to our privacy policy and terms of service.

Industry Insights

Discover insightful articles, industry insights, expert tips, and curated resources.

Machine Learning Recruitment Trends 2025 (UK): What Job Seekers Need To Know About Today’s Hiring Process

Summary: UK machine learning hiring has shifted from title‑led CV screens to capability‑driven assessments that emphasise shipped ML/LLM features, robust evaluation, observability, safety/governance, cost control and measurable business impact. This guide explains what’s changed, what to expect in interviews & how to prepare—especially for ML engineers, applied scientists, LLM application engineers, ML platform/MLOps engineers and AI product managers. Who this is for: ML engineers, applied ML/LLM engineers, LLM/retrieval engineers, ML platform/MLOps/SRE, data scientists transitioning to production ML, AI product managers & tech‑lead candidates targeting roles in the UK.

Why Machine Learning Careers in the UK Are Becoming More Multidisciplinary

Machine learning (ML) has moved from research labs into mainstream UK businesses. From healthcare diagnostics to fraud detection, autonomous vehicles to recommendation engines, ML underpins critical services and consumer experiences. But the skillset required of today’s machine learning professionals is no longer purely technical. Employers increasingly seek multidisciplinary expertise: not only coding, algorithms & statistics, but also knowledge of law, ethics, psychology, linguistics & design. This article explores why UK machine learning careers are becoming more multidisciplinary, how these fields intersect with ML roles, and what both job-seekers & employers need to understand to succeed in a rapidly changing landscape.

Machine Learning Team Structures Explained: Who Does What in a Modern Machine Learning Department

Machine learning is now central to many advanced data-driven products and services across the UK. Whether you work in finance, healthcare, retail, autonomous vehicles, recommendation systems, robotics, or consumer applications, there’s a need for dedicated machine learning teams that can deliver models into production, maintain them, keep them secure, efficient, fair, and aligned with business objectives. If you’re hiring for or applying to ML roles via MachineLearningJobs.co.uk, this article will help you understand what roles are typically present in a mature machine learning department, how they collaborate through project lifecycles, what skills and qualifications UK employers look for, what the career paths and salaries are, current trends and challenges, and how to build an effective ML team.