Be at the heart of actionFly remote-controlled drones into enemy territory to gather vital information.

Apply Now

Python Data Engineer

Tempest Vane Partners
London
2 days ago
Create job alert

The Client


This firm is a highly respected, technology-centric investment business operating across a broad range of asset classes. Their success is built on a mix of quantitative research, cutting-edge engineering and scalable data infrastructure. Engineers here play a central role: they design, build and maintain the platforms that underpin research, trading and large-scale data analysis.


It’s a collaborative environment where technical ownership is encouraged, engineering craft is valued, and impactful work directly supports sophisticated investment strategies.


What You'll Get


  • Work on the design and build of fast, scalable market-data systems used across trading and research groups.
  • Contribute to a modern engineering ecosystem: Python, cloud-native tooling, containerisation, large-scale data lake technologies.
  • Partner closely with exceptional quantitative researchers, data engineers and traders.
  • Influence architectural decisions and continuously refine pipeline performance.
  • Join a culture that values rigour, curiosity and continual improvement.
  • Benefit from strong compensation and long-term career growth within a high-performing engineering organisation.


Role Overview


  • Design, implement, and maintain high-throughput, low-latency pipelines for ingesting and processing tick-level market data at scale.
  • Operate and optimise timeseries databases (KDB, OneTick) to efficiently store, query, and manage granular datasets.
  • Architect cloud-native solutions for scalable compute, storage, and data processing, leveraging AWS, GCP, or Azure.
  • Develop and maintain Parquet-based data layers; contribute to evolving the data lake architecture and metadata management.
  • Implement dataset versioning and management using Apache Iceberg.
  • Collaborate closely with trading and quant teams to translate data requirements into robust, production-grade pipelines.
  • Implement monitoring, validation, and automated error-handling to ensure data integrity and pipeline reliability.
  • Continuously profile and optimise pipeline throughput, latency, and resource utilisation, particularly in latency-sensitive or HFT-like environments.
  • Maintain clear, precise documentation of data pipelines, architecture diagrams, and operational procedures.


What You Bring


  • 3+ years of software engineering experience, preferably focused on market-data infrastructure or quantitative trading systems.
  • Strong Python expertise with a solid grasp of performance optimisation and concurrency.
  • Proven experience designing, building, and tuning tick-data pipelines for high-volume environments.
  • Hands-on experience with Parquet storage; experience with Apache Iceberg or similar table formats is a plus.
  • Practical experience with containerisation (Docker) and orchestration platforms (Kubernetes).
  • Strong background in profiling, debugging, and optimising complex data workflows.
  • Experience with timeseries databases (KDB, OneTick) and/or performance-critical C++ components.
  • Deep understanding of financial markets, trading data, and quantitative workflows.
  • Excellent communication skills with the ability to articulate technical solutions to engineers and non-engineers alike.

Related Jobs

View all jobs

Python Data Engineer

Python Data Engineer

Senior Python Data Engineer — Experimentation Platform

Data Engineer (Python)

Senior Data Scientists/Data Engineers (Palantir, Python, Data Science)

Head of Data Engineering - Preston

Subscribe to Future Tech Insights for the latest jobs & insights, direct to your inbox.

By subscribing, you agree to our privacy policy and terms of service.

Industry Insights

Discover insightful articles, industry insights, expert tips, and curated resources.

Neurodiversity in Machine Learning Careers: Turning Different Thinking into a Superpower

Machine learning is about more than just models & metrics. It’s about spotting patterns others miss, asking better questions, challenging assumptions & building systems that work reliably in the real world. That makes it a natural home for many neurodivergent people. If you live with ADHD, autism or dyslexia, you may have been told your brain is “too distracted”, “too literal” or “too disorganised” for a technical career. In reality, many of the traits that can make school or traditional offices hard are exactly the traits that make for excellent ML engineers, applied scientists & MLOps specialists. This guide is written for neurodivergent ML job seekers in the UK. We’ll explore: What neurodiversity means in a machine learning context How ADHD, autism & dyslexia strengths map to ML roles Practical workplace adjustments you can ask for under UK law How to talk about neurodivergence in applications & interviews By the end, you’ll have a clearer sense of where you might thrive in ML – & how to turn “different thinking” into a genuine career advantage.

Machine Learning Hiring Trends 2026: What to Watch Out For (For Job Seekers & Recruiters)

As we move into 2026, the machine learning jobs market in the UK is going through another big shift. Foundation models and generative AI are everywhere, companies are under pressure to show real ROI from AI, and cloud costs are being scrutinised like never before. Some organisations are slowing hiring or merging teams. Others are doubling down on machine learning, MLOps and AI platform engineering to stay competitive. The end result? Fewer fluffy “AI” roles, more focused machine learning roles with clear ownership and expectations. Whether you are a machine learning job seeker planning your next move, or a recruiter trying to build ML teams, understanding the key machine learning hiring trends for 2026 will help you stay ahead.

Machine Learning Recruitment Trends 2025 (UK): What Job Seekers Need To Know About Today’s Hiring Process

Summary: UK machine learning hiring has shifted from title‑led CV screens to capability‑driven assessments that emphasise shipped ML/LLM features, robust evaluation, observability, safety/governance, cost control and measurable business impact. This guide explains what’s changed, what to expect in interviews & how to prepare—especially for ML engineers, applied scientists, LLM application engineers, ML platform/MLOps engineers and AI product managers. Who this is for: ML engineers, applied ML/LLM engineers, LLM/retrieval engineers, ML platform/MLOps/SRE, data scientists transitioning to production ML, AI product managers & tech‑lead candidates targeting roles in the UK.