Senior Software Engineer, AI Inference Systems

Germany
6 days ago
£70,000 – £120,000 pa

Salary

£70,000 – £120,000 pa

Job Type
Permanent
Work Pattern
Full-time
Work Location
Remote
Seniority
Senior
Posted
1 May 2026 (6 days ago)

We are seeking highly skilled and motivated software engineers to join us and build AI inference systems that serve large-scale models with extreme efficiency. You’ll architect and implement high-performance inference stacks, optimize GPU kernels and compilers, drive industry benchmarks, and scale workloads across multi-GPU, multi-node, and multi-cloud environments. You’ll collaborate across inference, compiler, scheduling, and performance teams to push the frontier of accelerated computing for AI.

What you’ll be doing:

  • Contribute features to vLLM that empower the newest models with the latest NVIDIA GPU hardware features; profile and optimize the inference framework (vLLM) with methods like speculative decoding, data/tensor/expert/pipeline-parallelism, prefill-decode disaggregation.

  • Develop, optimize, and benchmark GPU kernels (hand-tuned and compiler-generated) using techniques such as fusion, autotuning, and memory/layout optimization; build and extend high-level DSLs and compiler infrastructure to boost kernel developer productivity while approaching peak hardware utilization.

  • Define and build inference benchmarking methodologies and tools; contribute both new benchmark and NVIDIA’s submissions to the industry-leading MLPerf Inference benchmarking suite.

  • Architect the scheduling and orchestration of containerized large-scale inference deployments on GPU clusters across clouds.

  • Conduct and publish original research that pushes the pareto frontier for the field of ML Systems; survey recent publications and find a way to integrate research ideas and prototypes into NVIDIA’s software products.

What we need to see:

  • Bachelor’s degree (or equivalent expeience) in Computer Science (CS), Computer Engineering (CE) or Software Engineering (SE) with 7+ years of experience; alternatively, Master’s degree in CS/CE/SE with 5+ years of experience; or PhD degree with the thesis and top-tier publications in ML Systems, GPU architecture, or high-performance computing.

  • Strong programming skills in Python and C/C++; experience with Go or Rust is a plus; solid CS fundamentals: algorithms & data structures, operating systems, computer architecture, parallel programming, distributed systems, deep learning theories.

  • Knowledgeable and passionate about performance engineering in ML frameworks (e.g., PyTorch) and inference engines (e.g., vLLM and SGLang).

  • Familiarity with GPU programming and performance: CUDA, memory hierarchy, streams, NCCL; proficiency with profiling/debug tools (e.g., Nsight Systems/Compute).

  • Experience with containers and orchestration (Docker, Kubernetes, Slurm); familiarity with Linux namespaces and cgroups.

  • Excellent debugging, problem-solving, and communication skills; ability to excel in a fast-paced, multi-functional setting.

Ways to stand out from the crowd

  • Experience building and optimizing LLM inference engines (e.g., vLLM, SGLang).

  • Hands-on work with ML compilers and DSLs (e.g., Triton, TorchDynamo/Inductor, MLIR/LLVM, XLA), GPU libraries (e.g., CUTLASS) and features (e.g., CUDA Graph, Tensor Cores).

  • Experience contributing to containerization/virtualization technologies such as containerd/CRI-O/CRIU.

  • Experience with cloud platforms (AWS/GCP/Azure), infrastructure as code, CI/CD, and production observability.

  • Contributions to open-source projects and/or publications; please include links to GitHub pull requests, published papers and artifacts.

At NVIDIA, we believe artificial intelligence (AI) will fundamentally transform how people live and work. Our mission is to advance AI research and development to create groundbreaking technologies that enable anyone to harness the power of AI and benefit from its potential. Our team consists of experts in AI, systems and performance optimization. Our leadership includes world-renowned experts in AI systems who have received multiple academic and industry research awards. If you’re excited to build systems, kernels, and tools that make large-scale AI faster, more efficient, and easier to deploy, we’d love to hear from you.

#LI-Hybrid

Your base salary will be determined based on your location, experience, and the pay of employees in similar positions. For Poland: The base salary range is 292,500 PLN - 507,000 PLN for Level 4, and 375,000 PLN - 650,000 PLN for Level 5.

Related Jobs

View all jobs
Spotlight

Senior ML Compiler Engineer

Fractile Bristol, United Kingdom
Spotlight

Machine Learning Engineer - National Security (Gloucestershire)

Mind Foundry Gloucester, Gloucestershire, United Kingdom
On-site Clearance Required

Senior Software Engineer, AI Inference Systems

£70,000 – £120,000 pa Remote

Senior Software Engineer, AI Inference Systems

£70,000 – £120,000 pa Remote

Senior Software Engineer, AI Inference Systems

£70,000 – £120,000 pa Remote

Senior Software Engineer, AI Inference Systems

£70,000 – £120,000 pa Remote

Senior Software Engineer, AI Inference Systems

£70,000 – £120,000 pa Remote

Senior Software Engineer - AI Workbench

PhysicsX London, United Kingdom

Industry Insights

Discover insightful articles, industry insights, expert tips, and curated resources.

Where to Advertise Machine Learning Jobs in the UK (2026 Guide)

Advertising machine learning jobs in the UK requires a different approach to most technical hiring. The candidate pool is small, highly specialised and in demand across AI labs, financial services, healthcare, autonomous systems and consumer technology simultaneously. Machine learning engineers and researchers move between roles through professional networks, conference communities and specialist platforms — not general job boards where ML roles compete with unrelated software engineering positions for the same audience. This guide, published by MachineLearningJobs.co.uk, covers where to advertise machine learning roles in the UK in 2026, how the main platforms compare, what employers should expect to pay, and what the data says about hiring across different role types.

Machine Learning Jobs UK 2026: What to Expect Over the Next 3 Years

Machine learning has undergone a transformation that few technology disciplines can match. In the space of three years it has moved from a specialism sitting at the edges of most organisations' technology strategies to a capability that sits at the centre of them. The tools have changed, the expectations have shifted, and the range of industries treating machine learning as a core business function — rather than an experimental one — has expanded dramatically. For job seekers, this creates both opportunity and complexity in roughly equal measure. The machine learning jobs market of 2026 is significantly larger than it was three years ago, but it is also significantly more demanding. Employers have developed more sophisticated expectations, the technical bar for specialist roles has risen, and the landscape of tools, frameworks, and architectural patterns that practitioners are expected to know has broadened considerably. The candidates who will thrive over the next three years are those who understand where the discipline is heading — which specialisms are attracting the most investment, which technologies are reshaping what machine learning engineers and researchers are expected to build, and how the definition of a machine learning career is evolving beyond the model-building core toward a much wider range of roles across the full ML lifecycle. This article breaks down what the UK machine learning jobs market is likely to look like through to 2028 — covering the titles emerging right now, the technologies driving employer demand, the skills that will matter most, and how to position your career ahead of the curve.

New Machine Learning Employers to Watch in 2026: UK and Global Companies Driving ML Innovation

Machine learning (ML) has transitioned from a specialised field into a core business capability. In 2026, organisations across healthcare, finance, robotics, autonomous systems, natural language processing, and analytics are expanding their machine learning teams to build scalable intelligent products and services. For professionals exploring opportunities on www.MachineLearningJobs.co.uk , understanding the companies that are scaling, winning investment, or securing high‑impact contracts is crucial. This article highlights the new and high‑growth machine learning employers to watch in 2026, focusing on UK innovators, international firms with significant UK presence, and global platforms investing in machine learning talent locally.