Data Engineer (Databricks)

Pantheon
London
1 week ago
Create job alert

Pantheon has been at the forefront of private markets investing for more than 40 years, earning a reputation for providing innovative solutions covering the full lifecycle of investments, from primary fund commitments to co-investments and secondary purchases, across private equity, real assets and private credit.

Make your application after reading the following skill and qualification requirements for this position.We have partnered with more than 650 clients, including institutional investors of all sizes as well as a growing number of private wealth advisers and investors, with approximately $65bn in discretionary assets under management (as of December 31, 2023).Pantheon is undergoing a multi-year programme to build out a new best in class Data Platform using cloud native technologies hosted in Azure. We require an experienced and passionate hands-on Senior Data Engineer to design and implement new data pipelines for adaption to business and/or technology changes. This role will be integral to the success of this programme and establishing Pantheon as a data-centric organisation.You will be working with a modern Azure tech stack and proven experience of ingesting and transforming data from a variety of internal and external systems is core to the role.Key Responsibilities:Provide technical expertise in designing and implementing solutions, including determining root causes for complex data issues & developing practical efficient & permanent solutions.Carry out design, code, and test peer reviews throughout project providing technical guidance and to identify issues.Be a strong advocate for efficient code with extensive experience in delivering working solutions.Advocate and implement agile and lean principles ensuring efficiency within the development process.Collaborate with the Data QAs to implement appropriate testing, policies, and standards.Be a proactive and independent individual who drives quality within the organization.Essential Technical Skills:Experience in designing and developing data warehouse solutions.Building and configuring multistage Azure deployment pipelines.Advanced SQL, Python, PySpark.Azure Data Lake, Data Factory, Databricks and Functions.ARM / Terraform scripts to deploy infrastructure services.Experience in designing and building data ingestion and transformation pipelines, data lake, data warehouse / data marts etc.Experience working with Agile development methodologies such as Kanban/Scrum.A natural problem-solving attitude.Conversant with current industry and technology trends.Experience with dealing directly with stakeholders and business users.Desired Experience:Power BI.Business Objects Reporting.This job description is not to be construed as an exhaustive statement of duties, responsibilities, or requirements. You may be required to perform other job-related duties as reasonably requested by your manager.Pantheon is an Equal Opportunities employer, we are committed to building a diverse and inclusive workforce so if you're excited about this role but your past experience doesn't perfectly align we'd still encourage you to apply.

#J-18808-Ljbffr

Related Jobs

View all jobs

Data Engineer

Data Engineer

Data Engineer

Data Engineer

Data Engineer

Data Engineer

Get the latest insights and jobs direct. Sign up for our newsletter.

By subscribing you agree to our privacy policy and terms of service.

Industry Insights

Discover insightful articles, industry insights, expert tips, and curated resources.

Machine Learning Leadership for Managers: Strategies to Motivate, Mentor, and Set Realistic Goals in Data-Driven Teams

Machine learning (ML) has become an indispensable force in the modern business world, influencing everything from targeted marketing campaigns to advanced medical diagnostics. As industries integrate predictive algorithms and data-driven decision-making into their core operations, the need for effective leadership in machine learning environments has never been greater. Whether you’re overseeing a small team of data scientists or spearheading an enterprise-scale ML project, your leadership style must accommodate rapid innovation, complex problem-solving, and diverse stakeholder expectations. This guide provides actionable insights into how you can motivate, mentor, and establish achievable goals for your machine learning teams—ensuring they thrive in data-driven environments.

Top 10 Books to Advance Your Machine Learning Career in the UK

Machine learning (ML) remains one of the fastest-growing fields within technology, reshaping industries across the UK from finance and healthcare to e-commerce, telecommunications, and beyond. With increasing demand for ML specialists, job seekers who continually update their knowledge and skills hold a significant advantage. In this article, we've curated ten essential books every machine learning professional or aspiring ML engineer in the UK should read. Covering foundational theory, practical implementations, advanced techniques, and industry trends, these resources will equip you to excel in your machine learning career.

Navigating Machine Learning Career Fairs Like a Pro: Preparing Your Pitch, Questions to Ask, and Follow-Up Strategies to Stand Out

Machine learning (ML) has swiftly become one of the most in-demand skill areas across industries, with companies leveraging predictive models and data-driven insights to solve challenges in healthcare, finance, retail, manufacturing, and beyond. Whether you’re an early-career data scientist aiming to break into ML, a seasoned engineer branching into deep learning, or a product manager exploring AI-driven solutions, machine learning career fairs offer a powerful route to connect with prospective employers face-to-face. Attending these events can help you: Network with hiring managers and technical leads who make direct recruitment decisions. Gain insider insights on the latest ML trends and tools. Learn about emerging job roles and new industry verticals adopting machine learning. Showcase your interpersonal and communication skills, both of which are increasingly important in collaborative AI/ML environments. However, with many applicants vying for attention in a bustling hall, standing out isn’t always easy. In this detailed guide, we’ll walk you through how to prepare meticulously, pitch yourself confidently, ask relevant questions, and follow up effectively to land the machine learning opportunity that aligns with your ambitions.