DV Cleared - Data Engineer - ELK & NiFi

Defence
Worcester
5 days ago
Create job alert

Location:

Worcester Duration:

6 month initial contract Rate:

(Outside IR35) Security:

Active DV clearance required Role details: We are looking for 3 x Data Engineers to join our defence & security client on a contract basis. You will helpdesign, develop, and maintain secure and scalable data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi These roles are supporting our clients team in Worcester (fully onsite), and requires active UK DV clearance. Key Responsibilities: Design, develop, and maintain secure and scalable data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi Implement data ingestion, transformation, and integration processes, ensuring data quality and security Collaborate with data architects and security teams to ensure compliance with security policies and data governance standards Manage and monitor large-scale data flows in real-time, ensuring system performance, reliability, and data integrity Develop robust data models to support analytics and reporting within secure environments Perform troubleshooting, debugging, and performance tuning of data pipelines and the Elastic Stack Build dashboards and visualisations in Kibana to enable data-driven decision-making Ensure high availability and disaster recovery for data systems, implementing appropriate backup and replication strategies Document data architecture, workflows, and security protocols to ensure smooth operational handover and audit readiness.

What we are looking for in you: UK DV Clearance or the ability obtain it. Experience working as a Data Engineer in secure or regulated environments. Expertise in the Elastic Stack (Elasticsearch, Logstash, Kibana) for data ingestion, transformation, indexing, and visualization. Strong experience with Apache NiFi for building and managing complex data flows and integration processes. Knowledge of security practices for handling sensitive data, including encryption, anonymization, and access control. Familiarity with data governance, data quality management, and compliance standards in secure environments. Experience in managing large-scale, real-time data pipelines and ensuring their performance and reliability. Strong scripting and programming skills in Python, Bash, or other relevant languages. Working knowledge of cloud platforms (AWS, Azure, GCP) with a focus on data security and infrastructure as code. Excellent communication skills with the ability to collaborate effectively with cross-functional teams. Detail-oriented with a focus on ensuring data accuracy, quality, and security. Proactive problem-solving mindset and ability to troubleshoot complex data pipeline issues Desirable skills: Experience working in government, defence, or highly regulated industries with knowledge of relevant standards. Experience with additional data processing and ETL tools like Apache Kafka, Spark, or Hadoop. Familiarity with containerization and orchestration tools such as Docker and Kubernetes. Experience with monitoring and alerting tools such as Prometheus, Grafana, or ELK for data infrastructure. Understanding of ML algorithms, their development and implementation Confidence developing end-to-end solutions Experience with infrastructure as code e.g. Terraform, Ansible

If you are a skilled Data Systems Engineer with the required security clearance and expertise, we would love to hear from you. Apply now to join our client's high-performing team in Worcester.

TPBN1_UKTJ

Related Jobs

View all jobs

DV Cleared - Data Engineer - ELK & NiFi

Senior Data Engineer - DV Cleared

Project Manager

Senior Electronics Engineer

Science Analytics and Reporting Specialist

Get the latest insights and jobs direct. Sign up for our newsletter.

By subscribing you agree to our privacy policy and terms of service.

Industry Insights

Discover insightful articles, industry insights, expert tips, and curated resources.

Machine Learning Jobs in the Public Sector: Opportunities Across GDS, NHS, MOD, and More

Machine learning (ML) has rapidly moved from academic research labs to the heart of industrial and governmental operations. Its ability to uncover patterns, predict outcomes, and automate complex tasks has revolutionised industries ranging from finance to retail. Now, the public sector—encompassing government departments, healthcare systems, and defence agencies—has become an increasingly fertile ground for machine learning jobs. Why? Because government bodies oversee vast datasets, manage critical services for millions of citizens, and must operate efficiently under tight resource constraints. From using ML algorithms to improve patient outcomes in the NHS, to enhancing cybersecurity within the Ministry of Defence (MOD), there’s a growing demand for skilled ML professionals in UK public sector roles. If you’re passionate about harnessing data-driven insights to solve large-scale problems and contribute to societal well-being, machine learning jobs in the public sector offer an unparalleled blend of challenge and impact. In this article, we’ll explore the key reasons behind the public sector’s investment in ML, highlight the leading organisations, outline common job roles, and provide practical guidance on securing a machine learning position that helps shape the future of government services.

Contract vs Permanent Machine Learning Jobs: Which Pays Better in 2025?

Machine learning (ML) has swiftly become one of the most transformative forces in the UK technology landscape. From conversational AI and autonomous vehicles to fraud detection and personalised recommendations, ML algorithms are reshaping how organisations operate and how consumers experience products and services. In response, job opportunities in machine learning—including roles in data science, MLOps, natural language processing (NLP), computer vision, and more—have risen dramatically. Yet, as the demand for ML expertise booms, professionals face a pivotal choice about how they want to work. Some choose day‑rate contracting, leveraging short-term projects for potentially higher immediate pay. Others embrace fixed-term contract (FTC) roles for mid-range stability, or permanent positions for comprehensive benefits and a well-defined career path. In this article, we will explore these different employment models, highlighting the pros and cons of each, offering sample take‑home pay scenarios, and providing insights into which path might pay better in 2025. Whether you’re a new graduate with a machine learning degree or an experienced practitioner pivoting into an ML-heavy role, understanding these options is key to making informed career decisions.

Machine‑Learning Jobs for Non‑Technical Professionals: Where Do You Fit In?

The Model Needs More Than Math When ChatGPT went viral and London start‑ups raised seed rounds around “foundation models,” many professionals asked, “Do I need to learn PyTorch to work in machine learning?” The answer is no. According to the Turing Institute’s UK ML Industry Survey 2024, 39 % of advertised ML roles focus on strategy, compliance, product or operations rather than writing code. As models move from proof‑of‑concept to production, demand surges for specialists who translate algorithms into business value, manage risk and drive adoption. This guide reveals the fastest‑growing non‑coding ML roles, the transferable skills you may already have, real transition stories and a 90‑day action plan—no gradient descent necessary.