Job title: Data Engineer
Location: UK
Duration: 6-month contract Inside IR35
Working Mode: Fully Remote
Vetting: Active SC Clearance
Job Description:
Data Engineer Skills and Technologies Data Engineering:
- Strong understanding of data concepts
- data types, data structures, schemas (both JSON and Spark), schema management etc
- Strong understanding of complex JSON manipulation - Experience working with Data Pipelines using a custom Python/PySpark frameworks
- Strong understanding of the 4 core Data categories (Reference, Master, Transactional, Freeform) and the implications of each, particularly managing/handling Reference Data.
- Strong understanding of Data Security principles
- data owners, access controls
- row and column level, GDPR etc including experience of handling sensitive datasets
- Strong problem solving and analytical skills, particularly able to demonstrate these intuitively
Requirements
Required Skills:
Languages / Frameworks
- JSON
- YAML
- Python (as a programming language, not just able to write basic scripts. Pydantic experience would be a bonus.)
- SQL
- PySpark
- Delta Lake
- Bash (both CLI usage and scripting)
- Git
- Markdown
- Scala (bonus, not compulsory)
- Azure SQL Server as a HIVE Metastore (bonus) Technologies
- Azure Databricks
- Apache Spark
- Delta Tables
- Data processing with Python
- PowerBI (Integration / Data Ingestion)
- JIRA
The latest news, articles, and resources, sent to your inbox weekly.
Data Engineer | £60,000 - £65,000 | UK/Remote |Power BI | Synapse | Azure Data Bricks | ADF | SQL | Data Engineer | Reporting | Data Sets | Fabric |Are you a Data Engineer who enjoys getting involved with the architecture side of the role? Or maybe you want...
Opus Recruitment Solutions Sheffield
Role OverviewWe are looking for aData/BI Engineerto join our technology team and contribute to the development and maintenance of scalable data solutions. This role involves working with structured and unstructured data, integrating new sources, and optimizing data pipelines to support reporting and analytics.The position focuses onSQL Serveras the primary database...
Haystack Nottingham
Role OverviewWe are looking for aData/BI Engineerto join our technology team and contribute to the development and maintenance of scalable data solutions. This role involves working with structured and unstructured data, integrating new sources, and optimizing data pipelines to support reporting and analytics.The position focuses onSQL Serveras the primary database...
Haystack Glasgow
Role OverviewWe are looking for aData/BI Engineerto join our technology team and contribute to the development and maintenance of scalable data solutions. This role involves working with structured and unstructured data, integrating new sources, and optimizing data pipelines to support reporting and analytics.The position focuses onSQL Serveras the primary database...
Haystack Birmingham
Duration = 6 Months with likelihood of extending to 12Location = UK (Remote)Data EngineerDutiesWho you'll work withYou’ll be a member of a global team working on GenAI initiative. Clinet's Tech Ecosystem function is responsible for developing and delivering all technology solutions for the firm’s internal use. We are taking a...
eTeam Sheffield
Data Engineer - Snowflake | dbt | Python | AWSRemote (UK based) | 6-Month Contract | £450-£550 per day (Outside IR35)Are you aData Engineerlooking for your next contract opportunity? We're seeking an experienced professional to support aData Warehouse transformation, leveragingSnowflake, dbt, Python, and AWS.Key Responsibilities:Design, build, and optimisedata pipelineswithinSnowflake.Developdbtmodels and...
iO Associates - UK/EU Manchester
