Benchmarking Project Lead

October 9, 2026

Remote

Lead

Cambridge, GB

Indeed

Description

Apple is hiring a Benchmarking Project Lead for the Siri Evaluation team within the Siri organization. The role works on next-generation Apple Intelligence across iOS, iPadOS, macOS, watchOS and visionOS. It involves managing teams working on evaluation development and data science, and leading high-impact initiatives to bring state-of-the-art agentic evaluation to the whole Siri team. The position is based in Cambridge, England.

Responsibilities

Design and execute efficient data collection processes using humans in the loop
Lead annotation efforts for various languages and devices
Plan and manage the budget of annotator resourcing with Finance, Annotation Ops and International team partners
Track and improve the quality of human judgements through revision of annotator training materials, clear annotation questions, annotator training and efficient reviewing mechanisms
Collaborate with other engineering teams to design and build a tooling ecosystem for managing and browsing rich datasets
Manage teams working on evaluation development and data science
Lead high-impact initiatives to bring state-of-the-art agentic evaluation to the whole Siri team

Requirements

Experience as a senior manager driving evaluation efforts and leading teams
Managing teams working on evaluation development and data science
Agentic coding proficiency for data-science, data collection and visualisation tasks
Good understanding of metrics, crowd science, data collection, annotation analysis and statistics
Ability to work independently and cross-functionally to integrate partner team reporting systems and pipelines
Excellent communication skills and ability to thrive in a highly collaborative work environment
Attunement to computational linguistics, language quality and human-in-the-loop evaluation (preferred)
Good engineering practices for sustainable and easy-to-use data management pipelines (preferred)
Python experience and other tools for data collection and visualisation (preferred)

Similar jobs

Similar in meaning, not just matching keywords

This device does not meet the site requirements

What may not work

Benchmarking Project Lead

Job summary

Description

Responsibilities

Requirements

Similar jobs

Job summary

Similar jobs

Benchmarking Project Lead

Job summaryAnalysis and summary of the descriptionNeeds review [data-chart=chart-_R_39f1binpfiv6abqaqlb_] { --color-score: var(--destructive); } .dark [data-chart=chart-_R_39f1binpfiv6abqaqlb_] { --color-score: var(--destructive); } 30

Job summary

Description

Responsibilities

Requirements

Similar jobs

Job summaryAnalysis and summary of the descriptionNeeds review [data-chart=chart-_R_39f1binpfiv6abqaqlb_] { --color-score: var(--destructive); } .dark [data-chart=chart-_R_39f1binpfiv6abqaqlb_] { --color-score: var(--destructive); } 30

Job summary

Similar jobs