Engineering Manager - Data Ingest & Analysis (Model Dev Platform)
London, England
IT / Software / Data Analyst
#631281
Remote / WFH
Job Description
Wayve's efforts to develop our autonomous-driving AI starts with the Data Ingest & Analysis team. As its Engineering Manager, you will lead a multidisciplinary group that moves and processes hundreds of petabytes of multimodal driving data—the fuel that trains our end-to-end driving model. Without fast, reliable, and scaled access to driving data, the model cannot learn. Your leadership ensures every training job starts with the right bytes at the right time. Success in this role will have a far-reaching impact, ensuring that Wayve's has access to the data needed to train our AI model.
Challenges You Will Own
The Data Ingest & Analysis team own the very first step on Wayve's process of developing its autonomous driving software: the collect, transformation, and analysis of the hundreds of petabytes of driving data used to train our AI to drive vehicles. As the Engineering Manager of this team, you will:
Team Leadership and Strategic Oversight
Build and lead a team of data and platform engineers.
Define and execute the team's roadmap in collaboration with stakeholders from the AI and Vehicle Software organizations.
Set clear objectives, foster collaboration, and ensure timely delivery of projects.
Architecture of Petabyte Scale Data Processing
Manage and optimize large-scale Spark-based map-reduce data processing pipelines.
Oversee large scale transcoding and processing workflows to ensure data is ready for AI training.
Ingestion of Hundreds of Petabytes of Data
Enable the collection of driving data from a large number of both internal and third-party data sources
Normalize collected data to a consistent schema as needed by Wayve's AI training process
Ensure Reliable Operations of Workflow
Leverage work scheduling tools like Flyte to orchestrate multi-faceted data process workflow
Build and leverage operational observability to understand status of reliable operations
Set up processes to enable 24/7 reliable operations of workflow
Technical Mentorship and Talent Development
Recruit, mentor, and grow your team, cultivating technical excellence.