Data Engineer

The Role

As a Data Engineer, you’ll be responsible for designing, building, and maintaining robust data pipelines and architectures. You will work closely with stakeholders to understand complex data challenges, transform raw data into meaningful insights, and support analytics and reporting. This includes working with batch, streaming, real-time, and unstructured data, applying distributed compute techniques to handle large datasets efficiently.

Key Responsibilities

• Develop and maintain data ingestion pipelines and orchestration workflows
• Design database schemas and data models
• Integrate and enrich data from multiple sources, ensuring consistency and quality
• Design and implement ETL/ELT processes (e.g., using Apache NiFi)
• Produce reusable, maintainable code with a test-driven approach
• Maintain and enhance existing data platforms and services
• Investigate and resolve operational issues in integrated datasets
• Implement data security measures to protect sensitive information
• Support Agile delivery, breaking down user requirements into actionable tasks
• Monitor and optimise system performance for reliability and efficiency

Required Skills

• Apache Kafka
• Apache NiFi
• SQL and NoSQL databases (e.g., MongoDB)