Jobs

Senior Data Engineer


Job details
  • E1 EDF Trading Ltd
  • London
  • 4 months ago

Description

:

Department:Data ManagementData Insights team

The Data Management team plays a vital role in our organisation, transforming raw data into comprehensive and meaningful insights that can drive informed decision-making. Moreover the data management team is responsible for establishing and maintaining a robust and efficient data model that ensures data is consistent and accurate across the organisation.

Position purpose

The Senior Data Engineer is a key role in the Data Management team, dedicated to converting complex and large data sets into clear, actionable insights for the FO data-driven decision-making.

This role combines exceptional technical abilities in data modelling, transformation and visualisation with proficient skills on data processing technologies and methods. The data engineer is tasked with designing and executing effective data pipelines that support complex and interactive visualisations.

Candidates for this role should excel in event-driven architecture and advanced analytics modelling using Python and Dagster/Airflow, and possess in-depth experience with Tableau/Power BI and Alteryx, plus Data Warehouse management experience to ensure optimal database performance.

A significant aspect of this role includes engaging directly with business stakeholders and senior management to assess their requirements, suggest outcomes, and translate business objectives into effective data visualisation solutions while optimising the underlying data architecture.

Main responsibilities

Data Cleaning, Preparation and Transformation: Create data pipelines that combine internal and external EDFT data, across several APIs and databases (relational and non-relational) preparing and refining data for analysis.Data Quality Monitoring: Ensuring the accuracy and integrity of data at all times. This includes testing and validating data pipelines.Data Analysis, Design and Development of Visualisations and Dashboards: Creating clear, compelling, and accurate visualisations and dashboards that make complex data easily understandable.Database Architecture and Optimisation: Developing database structures for the Data Insights team that are both efficient and scalable, ensuring optimal performance.Stakeholder Management and Collaboration: Engaging directly with business stakeholders and senior management to assess their requirements, suggest outcomes, and translate business objectives into effective data visualisation solutions.Presentation of insights: Effectively presenting data visualisations and dashboards to stakeholders, transforming complex datasets into engaging, accessible stories, and ensuring that insights are not only informative but also resonate with the audience.

Experience and technical requirements

Strong Python (3.x) and SQL coding experience is essential. Experience with scripting languages (Bash, shell scripting), Proficiency with Git and CI/CD (ideally using Azure DevOps) Proficiency in data orchestration tools (Dagster/Airflow). Proficiency on ETL processes and data modelling (Alteryx). Proficiency in visualisation tools such as Tableau and Power BI. Proficiency with relational databases (SQL Server, Oracle or PostgreSQL). Strong skills in data analysis, including the use of statistical software and the ability to clean, process, and analyse large datasets.

Person Specification:

Excellent communication skills. Strong leadership and mentoring skills to guide and develop junior team members. Effective problem-solving and critical-thinking abilities to tackle data-related challenges. Keen eye for detail. Self-aware as to the value of current work vs. overall goal. Able to multitask, switch focus and prioritise own tasks being comfortable to work under pressure with demanding front office users. Takes ownership of any issues that come up and facilitates their resolution quickly using own initiative while managing expectations. A thirst for the latest technologies and automation coupled with a curiosity to research and innovate on new approaches. Interest in energy trading and willingness to work across the business on understanding the needs of different teams.

Beneficial Requirements:

Experience with Docker and containers desirable Experience with cloud computing (e.g. Microsoft Azure) beneficial Some experience of working in the energy market or prior exposure to a trading environment would be beneficial. Experience with real time streams (Kafka)and visualisations (Grafana and timeseries dbs) Experience interacting with the Tableau API is beneficial Experience interacting with the Alteryx API is beneficial. Experience interacting with the backend of Airflow/Dagster is beneficial.

Hours of work:

40 hours per week - Monday to Friday – Hybrid

Sign up for our newsletter

The latest news, articles, and resources, sent to your inbox weekly.

Similar Jobs

Senior Data Engineer

Senior Data EngineerLocation: Great Billing, Northampton, NN3 9EX.Salary: £60,000 to £70,000 DOE.Contract: Full time, permanent. 28 days per annum +3 bank holidays (Christmas, Boxing Day, NYD).Billing Finance is a family-owned vehicle financing Company based on the outskirts of Northampton. We focus on customers with non-standard credit profiles that may not...

Great Billing

Senior Data Engineer

Are you an experienced Data Engineer familiar with data extraction and modelling? Do you have SQL, Tableau and/or PowerBI skills? If so the following permanent role might be for you…My client is a privately owned business which operates in the scrap recycling sector who are looking to expand their talent...

Great Sankey

Senior Data Engineer - Azure - Hybrid - Manchester - £75k

Senior Data Engineer - Azure - Hybrid - Manchester - £75kThis is a fantastic opportunity for a senior data engineer to join a fast-growing IT consultancy based in Manchester! If you are an expert in the MS Azure tech stack, experienced overseeing a team of data professionals and building ETL...

Wilmslow

Senior Data Engineer

Senior Data Engineer - Global Insurance Organisation - Perm - Hybrid/CityJoin a leading global P&C focused insurance organisation as a Senior Data Engineer working with cutting edge MS tools to develop and distribute data to the business.Job DescriptionYou will be responsible for the optimisation and evolution of how a global...

City of London

Senior Data Engineer - Azure

I'm looking for an experienced Data Engineer to join a well-established organisation in Newcastle (hybrid-working), who truly recognise the value of leveraging their data to better understand their customers and to drive business growth.In this role you'll join a team of Data Engineers, responsible for the ongoing development of their...

Newcastle upon Tyne

Senior Data Engineer

Senior Data Engineer - Remote - £70-90K. Austin Fraser is currently recruiting for a Senior Data Engineer on behalf of an innovative and multinational client. You will be responsible for utilising your Data Engineering skills and experience in designing, implementing and supporting secure cloud-based data platforms. Furthermore, you’ll be working...

Manchester