Shape the Future of AIJoin one of the UK's fastest-growing companies and become a Professional Development Expert in Artificial Intelligence.

View Roles

Senior Data Engineer

Antler
London
1 week ago
Create job alert
About Prima Mente

Prima Mente’s goal is to deeply understand the brain, to protect the brain from neurological disease and enhance the brain in health. We do this by generating our own data, building brain foundation models, and translating discovery to real clinical and research impact.

Role focus - Biological Data Infrastructure at Petabyte Scale

Key Tasks:

  • Owning and scaling our data infrastructure by several orders of magnitude to handle > 100 petabyte-scale multi-omic datasets, including data pipelines, distributed data processing, and storage systems
  • Building a unified feature store for all our ML models and biological data analysis workflows
  • Efficiently storing and loading petabytes of data for ML bio data
  • Processing and storing predictions and evaluation metrics for large-scale biological forecasting and analysis models
  • Implementing data versioning and point-in-time correctness systems for evolving biological datasets
  • Building observable, debuggable data pipelines that handle the complexity of multi-omic data sources

Expected Growth

In 1 month you will be responsible for:

  • Analyzing current data infrastructure bottlenecks.
  • Implementing initial optimizations to existing pipelines.
  • Beginning work on scaling our feature store infrastructure for ML models.

In 3 months you directly own and have created:

  • Key components of our data processing systems.
  • Prototype streaming pipelines for real-time data ingestion.
  • Designs of our unified feature store architecture.

In 6 months you have implemented:

  • High-performance petabyte-scale data infrastructure.
  • Data versioning and point-in-time correctness systems.
  • Measurable improvements in data processing throughput and reliability.

Why Join Us:

  • Meaningful Impact: Contribute directly to research infrastructure that powers discoveries potentially impacting millions of lives.
  • Innovation & Autonomy: Work at the forefront of AI and multi-omics, with the freedom to propose and implement state-of-the-art infrastructure solutions.
  • Exceptional Team: Collaborate with talented colleagues from diverse backgrounds across ML, bioinformatics, and engineering.
  • Growth Opportunities: Continuous learning and growth opportunities in a rapidly advancing technical field.

Culture Insight

What we are doing is extremely hard. Prima Mente is for great people. We are team players who appreciate challenges, want to be hands-on, and thrive on curiosity by throwing away assumptions. We are focused on excellence at pace and huge personal growth. We are strong communicators who are highly disciplined and rigorous.

Prima Mente operates with a flat organizational structure. We gain and share knowledge by contributing to multiple opportunities. Leadership is given to those who show initiative and consistently deliver excellence.

We arrange our lives so we can work in person as much as possible.

Our Values

  • Exceptional performance at exceptional pace
    • The solutions we build demand uncompromising quality and rigour.
    • The problems we are solving are grave and present.
  • Inquisitive discovery
    • We embrace curiosity and creativity.
    • Every question is a path to a transformational breakthrough.
  • Radical candour
    • We practice unwavering honesty and transparency in all our challenges and interactions.
  • Purposeful individuality
    • Every individual in our team is celebrated for their identity, uniqueness, and experiences.
    • We are invested in each one’s bespoke personal development.
    • Nurturing individuality will supercharge our collective purpose and spirit.
  • Patient impact at scale
    • We have a steadfast commitment to improve the health and well-being of patients globally.
    • Every experiment run, every dataset analysed, and every innovation developed, is a step towards achieving a scalable impact.
Who You Are

You want to redefine what’s possible at the frontier of AI and biology. You’re intellectually curious, ambitious, and passionate about applying AI to biology. You thrive in interdisciplinary teams, possess an entrepreneurial spirit, and embrace the uncertainty and excitement of pioneering research.

Ideal experience

  • 4+ years of experience building data infrastructure or data platforms with demonstrated ability to solve complex distributed systems problems independently
  • Experience building infrastructure for large-scale data processing pipelines (both batch and streaming) using tools like Spark, Kafka, Apache Flink, Apache Beam, and with proprietary solutions like Nebius
  • Experience designing and implementing large-scale data storage systems (feature stores, timeseries DBs) for ML use cases, with strong familiarity with relational databases, data warehouses, object storage, and expertise in DB schema design
  • Experience with ML infrastructure and have worked at companies that use ML for core business functions
  • Experience building data pipelines for external data sources that are observable, debuggable, and verifiably correct, having dealt with challenges like data versioning, point-in-time correctness, and evolving schemas
  • Strong distributed systems and infrastructure skills - comfortable scaling and debugging Kubernetes services, writing Terraform, and working with orchestration tools like Flyte, Airflow, or Temporal
  • Experience with cloud platforms (AWS, GCP, Azure) and container technologies
  • Strong software engineering skills with ability to write easy-to-extend and well-tested code
  • Excellent communication skills and experience collaborating within multidisciplinary teams
  • Comfortable with ambiguity and a fast-moving environment, with a bias for action
  • Learn and pick up new skills quickly
  • Familiarity with bioinformatics or biological data handling (this will be supported by our in-house bioinformatics team)
  • Knowledge of data governance, compliance, and security standards relevant to healthcare or biotech

Interview Process

Our interview process is hard from the beginning, so please do come prepared to show us your strongest self. Marie is based in SF and Hannah in London - we are both available to support this process.

We promise to communicate clearly about our process, look for your strengths, be transparent in our feedback and listen to your feedback - we are always learning.

The interview steps are listed below. 1-3 are done remotely over video call. Our preference for 4-7 is in person, but remote is possible too. At stage 4 more information will be shared about the following steps.

  • Screen with Marie or Hannah
  • Meet Ravi
  • CV Deep Dive
  • Take Home Technical Challenge & Discussion
  • Analysis Challenge with Ravi
  • Systems Design & Live Coding
  • Presentation of your work to the wider team


#J-18808-Ljbffr

Related Jobs

View all jobs

Senior Data Engineer

Senior Data Engineer

Senior Data Engineer

Senior Data Engineer - Winchester

Senior Data Engineer - Cloud

Senior Data Engineer

Subscribe to Future Tech Insights for the latest jobs & insights, direct to your inbox.

By subscribing, you agree to our privacy policy and terms of service.

Industry Insights

Discover insightful articles, industry insights, expert tips, and curated resources.

Automate Your Machine Learning Jobs Search: Using ChatGPT, RSS & Alerts to Save Hours Each Week

ML jobs are everywhere—product companies, labs, consultancies, fintech, healthtech, robotics—often hidden in ATS portals or duplicated across boards. The fastest way to stay on top of them isn’t more scrolling; it’s automation. With keyword-rich alerts, RSS feeds, and a reusable ChatGPT workflow, you can bring relevant roles to you, triage them in minutes, and tailor strong applications without burning your evenings. This is a copy-paste playbook for www.machinelearningjobs.co.uk readers. It’s UK-centric, practical, and designed to save you hours each week. What You’ll Have Working In 30 Minutes A role & keyword map spanning LLM/NLP, Vision, Core ML, Recommenders, MLOps/Platform, Research/Applied Science, and Edge/Inference optimisation. Shareable Boolean searches you can paste into Google & job boards to cut noise. Always-on alerts & RSS feeds delivering fresh roles to your inbox/reader. A ChatGPT “ML Job Scout” prompt that deduplicates, scores fit, and outputs tailored actions. A lightweight pipeline tracker so deadlines and follow-ups never slip.

10 Machine‑Learning Recruitment Agencies in the UK You Should Know (2025 Job‑Seeker Guide)

With deep‑learning projects now integral across healthcare, finance and tech, UK demand for machine‑learning talent is booming. Lightcast shows +50 % YoY growth in UK adverts referencing “machine learning,” “deep learning,” “computer vision” or “reinforcement learning” in Q1 2025. Monthly vacancies sit around 1,800–2,100, but certified ML specialists number fewer than 15,000. Specialist recruiters help candidates access hidden roles, competitive packages, and structured interview prep. How we screened: Only UK‑registered agencies with clear ML/AI or Data practices Agencies that posted ≥ 5 UK ML roles between March and June 2025

Machine Learning Jobs Skills Radar 2026: Emerging Tools, Frameworks & Platforms to Learn Now

Machine learning is no longer confined to academic research—it's embedded in how UK companies detect fraud, recommend content, automate processes & forecast risk. But with model complexity rising and LLMs transforming workflows, employers are demanding new skills from machine learning professionals. Welcome to the Machine Learning Jobs Skills Radar 2026—your annual guide to the top languages, frameworks, platforms & tools shaping machine learning roles in the UK. Whether you're an aspiring ML engineer or a mid-career data scientist, this radar shows what to learn now to stay job-ready in 2026.