Lead Data Engineer, Data Reliability

Disney Cruise Line - The Walt Disney Company
London
1 week ago
Create job alert

Disney Entertainment & ESPN Product & Technology

On any given day at Disney Entertainment & ESPN Product & Technology (DEEP&T), we’re reimagining ways to create magical viewing experiences for the world’s most beloved stories while also transforming Disney’s media business for the future. Whether that’s evolving our streaming and digital products in new and immersive ways, powering worldwide advertising and distribution to maximize flexibility and efficiency, or delivering Disney’s unmatched entertainment and sports content, every day is a moment to make a difference to partners and to hundreds of millions of people around the world.

A few reasons why we think you’d love working for Disney Entertainment & ESPN Technology

  • Building the future of Disney’s media business:DE&E Technologists are designing and building the infrastructure that will power Disney’s media, advertising, and distribution businesses for years to come.

  • Reach & Scale:The products and platforms this group builds and operates delight millions of consumers every minute of every day – from Disney+ and Hulu, to ABC News and Entertainment, to ESPN and ESPN+, and much more.

  • Innovation:We develop and execute groundbreaking products and techniques that shape industry norms and enhance how audiences experience sports, entertainment & news.

About The Team & Role

The Data Reliability Engineering team for Disney’s Product and Data Engineering team is responsible for maintaining and improving the reliability of Disney Entertainment’s big data platform, which processes hundreds of terabytes of data and billions of events daily.

The Lead Data Engineer will help us in the ongoing mission of delivering outstanding services to our users allowing Disney Entertainment to be more data-driven. You will work closely with our partner teams to monitor and drive improvements for reliability and observability of their critical data pipelines and deliverables. This is a high-impact role where your work informs decisions affecting millions of consumers, with a direct tie to The Walt Disney Company’s revenue. You will be making an outsized impact in an organization that values data as its top priority. We are a tight and driven team with big goals, so we seek people who are passionate about solving the toughest challenges and working at scale, using, supporting, and building distributed systems in a fast-paced collaborative team environment. We also support a healthy work-life-balance.

Responsibilities

  • Assist in designing and developing a platform to support incident observability and automation. This team will be required to build high quality data models and products that monitor and reports on data pipeline health and data quality.

  • Lead project work efforts internally and externally setting project deliverables, review design documents, perform code reviews and help mentor junior members of the team.

  • Collaborate with engineering teams to improve, maintain, performance tune, and respond to incidents on our big data pipeline infrastructure.

  • Own building out key components for observability and intelligent monitoring of data pipelines and infrastructure to achieve early and automated anomaly detection and alerting. Present your research and insights to all levels of the company, clearly and concisely.

  • Build solutions to continually improve our software release and change management process using industry best practices to help DSS maintain legal compliance.

Basic Qualifications

  • 7+ years experience working on mission critical data pipelines and ETL systems, hands-on experience with big data technology, systems and tools such as AWS, Hadoop, Hive, and Snowflake

  • Detailed problem-solving approach, coupled with a strong sense of ownership and drive

  • A passionate bias to action and passion for delivering high-quality data solutions

  • Expertise with common Data Engineering languages such as Python, Scala, Java, SQL and a proven ability to learn new programming languages

  • Experience with workflow orchestration tools such as Airflow

  • Deep understanding of end-to-end pipeline design and implementation.

  • Attention to detail and quality with excellent problem solving and interpersonal skills

Preferred Qualifications

  • Advanced degrees are a plus.

  • Strong data visualizations skills to convey information and results clearly

  • Ability to work independently and drive your own projects.

  • Exceptional interpersonal and communication skills.

  • Impactful presentation skills in front of a large and diverse audience.

  • Experience with DevOps tools such as Docker, Kubernetes, Jenkins, etc.

  • Innate curiosity about consumer behavior and technology

  • Experience with event messaging frameworks like Apache Kafka

  • A fan of movies and television is a strong plus.

Required Education

  • Bachelor’s degree in Computer Science, Information Systems, Software, Electrical or Electronics Engineering, or comparable field of study, and/or equivalent work experience.

Additional Information

#DISNEYTECH


The hiring range for this position in Santa Monica, California is $152,200 to $204,100 per year, in Bristol, Connecticut is $152,200 to $204,100 per year, in Seattle, Washington is $159,500 to $213,900 per year. in New York City, NY is $159,500 to $213,900 per year, in San Francisco, California is $166,800 to $223,600 per year. The hiring range for this remote position is $145,000 to $223,600 per year, which factors in various geographic regions.The base pay actually offered will take into account internal equity and also may vary depending on the candidate’s geographic region, job-related knowledge, skills, and experience among other factors. A bonus and/or long-term incentive units may be provided as part of the compensation package, in addition to the full range of medical, financial, and/or other benefits, dependent on the level and position offered.J-18808-Ljbffr

Related Jobs

View all jobs

Lead Machine Learning Engineer, Associate Director, London

Software Engineer

Staff Machine Learning Engineer Melbourne

Data Engineering Lead (Snowflake & AWS Environment)

Data Engineering Lead (Snowflake & AWS Environment)

Data Engineering Lead - Finance and Master

Get the latest insights and jobs direct. Sign up for our newsletter.

By subscribing you agree to our privacy policy and terms of service.

Industry Insights

Discover insightful articles, industry insights, expert tips, and curated resources.

Portfolio Projects That Get You Hired for Machine Learning Jobs (With Real GitHub Examples)

In today’s data-driven landscape, the field of machine learning (ML) is one of the most sought-after career paths. From startups to multinational enterprises, organisations are on the lookout for professionals who can develop and deploy ML models that drive impactful decisions. Whether you’re an aspiring data scientist, a seasoned researcher, or a machine learning engineer, one element can truly make your CV shine: a compelling portfolio. While your CV and cover letter detail your educational background and professional experiences, a portfolio reveals your practical know-how. The code you share, the projects you build, and your problem-solving process all help prospective employers ascertain if you’re the right fit for their team. But what kinds of portfolio projects stand out, and how can you showcase them effectively? This article provides the answers. We’ll look at: Why a machine learning portfolio is critical for impressing recruiters. How to select appropriate ML projects for your target roles. Inspirational GitHub examples that exemplify strong project structure and presentation. Tangible project ideas you can start immediately, from predictive modelling to computer vision. Best practices for showcasing your work on GitHub, personal websites, and beyond. Finally, we’ll share how you can leverage these projects to unlock opportunities—plus a handy link to upload your CV on Machine Learning Jobs when you’re ready to apply. Get ready to build a portfolio that underscores your skill set and positions you for the ML role you’ve been dreaming of!

Machine Learning Job Interview Warm‑Up: 30 Real Coding & System‑Design Questions

Machine learning is fuelling innovation across every industry, from healthcare to retail to financial services. As organisations look to harness large datasets and predictive algorithms to gain competitive advantages, the demand for skilled ML professionals continues to soar. Whether you’re aiming for a machine learning engineer role or a research scientist position, strong interview performance can open doors to dynamic projects and fulfilling careers. However, machine learning interviews differ from standard software engineering ones. Beyond coding proficiency, you’ll be tested on algorithms, mathematics, data manipulation, and applied problem-solving skills. Employers also expect you to discuss how to deploy models in production and maintain them effectively—touching on MLOps or advanced system design for scaling model inferences. In this guide, we’ve compiled 30 real coding & system‑design questions you might face in a machine learning job interview. From linear regression to distributed training strategies, these questions aim to test your depth of knowledge and practical know‑how. And if you’re ready to find your next ML opportunity in the UK, head to www.machinelearningjobs.co.uk—a prime location for the latest machine learning vacancies. Let’s dive in and gear up for success in your forthcoming interviews.

Negotiating Your Machine Learning Job Offer: Equity, Bonuses & Perks Explained

How to Secure a Compensation Package That Matches Your Technical Mastery and Strategic Influence in the UK’s ML Landscape Machine learning (ML) has rapidly shifted from an emerging discipline to a mission-critical function in modern enterprises. From optimising e-commerce recommendations to powering autonomous vehicles and driving innovation in healthcare, ML experts hold the keys to transformative outcomes. As a mid‑senior professional in this field, you’re not only crafting sophisticated algorithms; you’re often guiding strategic decisions about data pipelines, model deployment, and product direction. With such a powerful impact on business results, companies across the UK are going beyond standard salary structures to attract top ML talent. Negotiating a compensation package that truly reflects your value means looking beyond the numbers on your monthly payslip. In addition to a competitive base salary, you could be securing equity, performance-based bonuses, and perks that support your ongoing research, development, and growth. However, many mid‑senior ML professionals leave these additional benefits on the table—either because they’re unsure how to negotiate them or they simply underestimate their long-term worth. This guide explores every critical aspect of negotiating a machine learning job offer. Whether you’re joining an AI-focused start-up or a major tech player expanding its ML capabilities, understanding equity structures, bonus schemes, and strategic perks will help you lock in a package that matches your technical expertise and strategic influence. Let’s dive in.