AIML - Sr Machine Learning Platform Engineer, Evaluation

Summary We are seeking a highly skilled and experienced machine learning engineer to join AIML Evaluation to build the systems that evaluate and refine Apple's foundation models and agents. As a key member of the team, you will help design and develop benchmarks, evaluators, simulation environments, and prompt and context optimization pipelines that drive quality improvements across Apple's AI experiences. You will collaborate with product teams and the foundation model team to close the loop between observation and improvement, contributing datasets, environments, and reward signals that drive model and agent quality.

Description Most model evaluation stops at a scorecard. The failing traces get thrown away, even though they're the most relevant training signal we have. Our team puts them to work. We evaluate across offline benchmarks, device-in-the-loop simulation, and on-device observation in production, and we turn what we find into persona-grounded datasets and reward signals for foundation model post-training.

In this role, you will build the systems that connect failure analysis to measurable model and agent improvements across a broad range of AI products at Apple. The path from finding a problem to fixing it is short: the loop you build finds the failure, produces the data and checks the result. The work reaches millions of users.

Responsibilities

Designing and building evaluation infrastructure for agents and foundation models. Setting technical direction for the evaluation system and bringing partner teams along. Developing LLM judges, reward models calibrated against human feedback. Developing prompt and context optimization pipelines. Building and integrating simulation environments for agent evaluation and trajectory-based data generation. Working with product and research teams to identify, prioritize, and address quality gaps. Building shared foundations such as run lineage, a run registry, and MCP interfaces for client teams' agents.

Minimum Qualifications Strong background in machine learning and distributed systems. A proven track record of solving ambiguous problems end to end: defining the problem and the metric, building the solution, and showing measured impact. Experience leading technical work across multiple codebases, teams, and organizations. Experience building and maintaining ML infrastructure for evaluation, training, or deployment. 6+ years of professional experience as a software engineer, preferably in machine learning or a related field. Bachelor's or Master's degree in Computer Science or a related field, or equivalent practical experience.

Preferred Qualifications Experience with LLM evaluation, LLM-as-judge, or reward modeling. Experience with prompt optimization, agent harness development, or post-training (SFT, DPO, RLHF). Proficiency in Python and ML frameworks such as PyTorch or JAX. Experience with agentic systems, simulation environments, or trajectory-based data generation. Familiarity with on-device or privacy-preserving ML. Understanding of how training data is used in post-training and how data choices affect model behavior. Excellent communication skills.

Pay & Benefits At Apple, base pay is one part of our total compensation package and is determined within a range. This provides the opportunity to progress as you grow and develop within a role. The base pay range for this role is between $175,000 and $308,500, and your base pay will depend on your skills, qualifications, experience, and location.

Apple employees also have the opportunity to become an Apple shareholder through participation in Apple’s discretionary employee stock programs. Apple employees are eligible for discretionary restricted stock unit awards, and can purchase Apple stock at a discount if voluntarily participating in Apple’s Employee Stock Purchase Plan. You’ll also receive benefits including: Comprehensive medical and dental coverage, retirement benefits, a range of discounted products and free services, and for formal education related to advancing your career at Apple, reimbursement for certain educational expenses - including tuition. Additionally, this role might be eligible for discretionary bonuses or commission payments as well as relocation. Learn more about Apple Benefits

Note: Apple benefit, compensation and employee stock programs are subject to eligibility requirements and other terms of the applicable plan or program.

EEO Statement Apple is an equal opportunity employer that is committed to inclusion and diversity. We seek to promote equal opportunity for all applicants without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, Veteran status, or other legally protected characteristics. Learn more about your EEO rights as an applicant

Accessibility At Apple, we believe accessibility is a fundamental human right. You’ll find that idea reflected in everything here - in our culture, our benefits and our digital tools. By welcoming as many perspectives as possible, we help you build a career where you feel like you belong.

Learn about accessibility in Apple’s workplace

Learn about reasonable accommodations for job applicants

Application Deadline Apple accepts applications to this posting on an ongoing basis.