Senior Data Engineer

Our client is redefining electronic trading by developing high-performance, AI-powered solutions that enable smarter, faster decision-making in dynamic markets. Our client is looking for a Senior Data Engineer to be a part of a forward-thinking team in London, working at the intersection of AI and trading technology. You'll collaborate with leading researchers, data scientists, and engineers to build scalable data infrastructure that supports advanced analytics and machine learning. This is a greenfield opportunity with full ownership from design to deployment, allowing you to shape systems that directly impact trading and AI capabilities.

In this role, you'll architect and maintain robust data pipelines for ingesting, transforming, and storing large volumes of structured and unstructured data, including time-series datasets. You'll automate ETL processes, implement validation frameworks, and ensure high data quality and system reliability. You'll also support AI model development by optimising data retrieval, integrating scalable APIs, and working with vector databases such as FAISS and Pinecone. Familiarity with MLOps, LLM workflows, and RAG frameworks is essential, along with experience deploying AI/ML models into production environments. Your work will be critical to enabling rapid experimentation, research validation, and the seamless transition of prototypes into production.

Key Responsibilities:

• Develop and maintain scalable pipelines for ingesting, transforming, and storing large volumes of data.
• Design systems for handling text-heavy and time-series data, ensuring performance and reliability.
• Collaborate with data scientists to optimise data retrieval and training workflows for AI models.
• Automate ETL processes and improve data quality through validation and monitoring tools.
• Support experimentation and deployment by building flexible, production-ready data systems.
• Ensure strong documentation, coding standards, and data governance practices.
• Support AI research and production systems, collaborating with Principal data scientist to operationalise RAG systems
• Architect and implement data ingestion, transformation, storage, and retrieval systems, ensuring they are resilient, high-performing, and fit for future growth.

Requirements:

• 5+ years in data engineering with strong SQL and cloud data platform experience.
• Skilled in handling unstructured data and NLP datasets.
• Familiar with MLOps, RAG frameworks, and deploying AI/ML models into production.
• Experience implementing APIs, vector databases (e.g., FAISS, Pinecone), and LLM pipelines.
• Knowledge of data security, versioning tools (e.g., DVC, MLflow), and time-series databases.