Principal Machine Learning Infrastructure Engineer

PhysicsX
London, United Kingdom
5 days ago
Seniority
Lead
Posted
14 Apr 2026 (5 days ago)

About us

PhysicsX is a deep-tech company with roots in numerical physics and Formula One, dedicated to accelerating hardware innovation at the speed of software. We are building an AI-driven simulation software stack for engineering and manufacturing across advanced industries. By enabling high-fidelity, multi-physics simulation through AI inference across the entire engineering lifecycle, PhysicsX unlocks new levels of optimization and automation in design, manufacturing, and operations — empowering engineers to push the boundaries of possibility. Our customers include leading innovators in Aerospace & Defense, Materials, Energy, Semiconductors, and Automotive.

Note:We are currently recruiting for multiple positions, however please only apply for the role that best aligns with your skillset and career goals.

The Role

The Principal ML Infrastructure Engineer will extend and operate the infrastructure that powers our research model training, fine-tuning, and serving pipelines. You will be embedded within our Research function, partnering directly with ML engineers and research scientists to ensure they can train Large Physics Models efficiently and reliably at scale.

Team Context

In this role, you will be vertically embedded in Research, working daily with:

  • Research Scientists who determine the model architectures and methods
  • ML Engineers who implement and develop the models
  • Simulation Data Engineers who are accountable for upstream data pipelines

You will have end-to-end responsibilities over the research infrastructure, with the autonomy to make architectural decisions and the responsibility to keep data flowing reliably.

Horizontally, you will be part of an infrastructure engineering group responsible for infrastructure across the company.

What you will do

Training Infrastructure

  • Design and operate distributed training infrastructure for neural operator architectures (Transolver, Point Cloud Transformer, etc.) on our large NVIDIA DGX B200 platform.
  • Optimize training pipelines for throughput, fault tolerance, and cost efficiency, including checkpointing strategies, gradient accumulation, and multi-node synchronization.
  • Build and maintain experiment tracking and observability systems that give researchers clear visibility into training runs, hyperparameter sweeps, and model performance.

Data I/O and Performance

  • Solve data loading bottlenecks for large-scale mesh datasets.
  • Optimize data pipelines for efficient I/O from cloud storage, including prefetching, caching, and format optimization.
  • Work with heterogeneous data sources of varying formats and resolutions.

Model Serving and Deployment

  • Build serving infrastructure for pre-trained LPMs, supporting both zero-shot inference and uncertainty quantification (Monte Carlo Dropout).
  • Design and implement model packaging pipelines for customer deployment. Models must run reliably in customer environments with fine-tuning capabilities.
  • Ensure reproducibility: any model checkpoint should be deployable with consistent behaviour.

Platform and Tooling

  • Improve developer experience for the Research team with fast iteration cycles, reliable CI/CD, clear debugging tools.
  • Collaborate with the broader Infrastructure team on shared patterns and standards.

What you bring to the table

  • Ability to scope and effectively deliver projects, prioritising activity as needed.
  • Problem-solving skills and the ability to analyse issues, identify causes, and recommend solutions quickly.
  • Excellent collaboration and communication skills,especially in a research setting. You can translate "the model isn't converging" into infrastructure hypotheses and solutions, and can bridge technical abstractions with implementations.
  • 5+ years of experience building and operating ML infrastructure at scale:
    • Deep expertise in distributed training: you've debugged NCCL hangs, optimized collective communication, and know when to use FSDP vs. DDP vs. pipeline parallelism
    • Strong systems fundamentals: Linux, networking (including domain specific NVLink and InfiniBand), storage I/O, profiling and performance optimization
    • Production experience with Kubernetes and SLURM for job orchestration on GPU clusters
    • Proficiency in Python and ML frameworks (PyTorch strongly preferred)
    • Experience with cloud GPU infrastructure; ideally CoreWeave or similar GPU/HPC-focused clouds

Ideally

  • Experience with geometric deep learning or neural operators, ****architectures that operate on meshes, point clouds, or graphs
  • Background in HPC for simulation engineering, familiarity with how CFD/FEA workflows generate and consume data
  • Experience building model serving infrastructure with latency and throughput requirements
  • Familiarity with experiment tracking tools (Weights & Biases, MLflow) and observability stacks (Prometheus, Grafana)
  • Experience packaging models for deployment into customer environments (containers, model registries, versioning)

What we offer

Build what actually matters

Help shape an AI-native engineering company at a formative stage, tackling problems that genuinely matter for industry and society. This is work with real-world impact - and something you can be proud to stand behind.

Learn alongside exceptional people

Work with a high-caliber, collaborative team of engineers, scientists, and operators who care deeply about doing great work, and about helping each other get better. We come from diverse backgrounds, but we share a commitment to operating at the highest level and addressing some of the most complex challenges out there. If you’re ambitious, thoughtful, and driven by impact, you’ll feel at home.

Influence over hierarchy

We operate with a flat structure: good ideas win - wherever they come from. Questioning assumptions and challenging the status quo isn’t just welcomed, it’s expected.

Sustainable pace, long-term ambition

Building meaningful technology is a marathon, not a sprint. We believe in balancing focused, ambitious work with a life beyond it. Our hybrid model blends time together in our Shoreditch office with work-from-home days, giving you the flexibility to work sustainably while staying connected in person.

And it doesn’t stop there …

🚀Equity options - share meaningfully in the company you’re helping to build.

🏦10% employer pension contribution - because investing in future matters.

🍽️Free office lunches - to keep you energised and focused.

👶Enhanced parental leave - 3 months full pay paternity and 6 months full pay maternity leave, to provide extra flexibility during the moments that matter most.

🍼YellowNest nursery scheme - to help working parents manage childcare costs.

☀️ 25 days of Annual Leave (+ Public Holidays) - because taking time to rest matters.

🏥Private medical insurance - 100% employee cover, giving you complete peace of mind.

💪Wellhub Subscription - gain access to thousands of gyms, classes and wellness apps, supporting both physical and mental wellbeing.

👀Eye tests - because good work depends on good health.

📈Personal development - dedicated support for learning, development, and leveling up over time.

💛Employee Assistance Programme (EAP) - confidential wellbeing support, available whenever you need it.

🚲Bike2Work scheme and 🚆Season ticket loan - to make getting to work easier and greener.

🚗Octopus EV salary sacrifice - for a simpler, more sustainable way to drive electric.

🔎 Watch this space, we’re continuing to build this as we grow…

We value diversity and are committed to equal employment opportunity regardless of sex, race, religion, ethnicity, nationality, disability, age, sexual orientation or gender identity. We strongly encourage individuals from groups traditionally underrepresented in tech to apply. To help make a change, we sponsor bright women from disadvantaged backgrounds through their university degrees in science and mathematics. We collect diversity and inclusion data solely for the purpose of monitoring the effectiveness of our equal opportunities policies and ensuring compliance with UK employment and equality legislation. This information is confidential, used only in aggregate form, and will not influence the outcome of your application.

Related Jobs

View all jobs

Senior Software Engineer, ML Platform (Stability & Infrastructure)

Isomorphic Labs London, United Kingdom

Principal Software Engineer - Engineering Applications

PhysicsX London, United Kingdom

ML Research Engineer, London

Isomorphic Labs London, United Kingdom

Senior Machine Learning Engineer

Faculty London, United Kingdom
Hybrid

Machine Learning Engineer

Faculty London, United Kingdom
Hybrid

Senior Security Engineer

Isomorphic Labs United Kingdom

Subscribe to Future Tech Insights for the latest jobs & insights, direct to your inbox.

By subscribing, you agree to our privacy policy and terms of service.

Industry Insights

Discover insightful articles, industry insights, expert tips, and curated resources.

Where to Advertise Machine Learning Jobs in the UK (2026 Guide)

Advertising machine learning jobs in the UK requires a different approach to most technical hiring. The candidate pool is small, highly specialised and in demand across AI labs, financial services, healthcare, autonomous systems and consumer technology simultaneously. Machine learning engineers and researchers move between roles through professional networks, conference communities and specialist platforms — not general job boards where ML roles compete with unrelated software engineering positions for the same audience. This guide, published by MachineLearningJobs.co.uk, covers where to advertise machine learning roles in the UK in 2026, how the main platforms compare, what employers should expect to pay, and what the data says about hiring across different role types.

New Machine Learning Employers to Watch in 2026: UK and Global Companies Driving ML Innovation

Machine learning (ML) has transitioned from a specialised field into a core business capability. In 2026, organisations across healthcare, finance, robotics, autonomous systems, natural language processing, and analytics are expanding their machine learning teams to build scalable intelligent products and services. For professionals exploring opportunities on www.MachineLearningJobs.co.uk , understanding the companies that are scaling, winning investment, or securing high‑impact contracts is crucial. This article highlights the new and high‑growth machine learning employers to watch in 2026, focusing on UK innovators, international firms with significant UK presence, and global platforms investing in machine learning talent locally.

How Many Machine Learning Tools Do You Need to Know to Get a Machine Learning Job?

Machine learning is one of the most exciting and rapidly growing areas of tech. But for job seekers it can also feel like a maze of tools, frameworks and platforms. One job advert wants TensorFlow and Keras. Another mentions PyTorch, scikit-learn and Spark. A third lists Mlflow, Docker, Kubernetes and more. With so many names out there, it’s easy to fall into the trap of thinking you must learn everything just to be competitive. Here’s the honest truth most machine learning hiring managers won’t say out loud: 👉 They don’t hire you because you know every tool. They hire you because you can solve real problems with the tools you know. Tools are important — no doubt — but context, judgement and outcomes matter far more. So how many machine learning tools do you actually need to know to get a job? For most job seekers, the real number is far smaller than you think — and more logically grouped. This guide breaks down exactly what employers expect, which tools are core, which are role-specific, and how to structure your learning for real career results.