AI Software Evaluation Engineer, Siri AI Quality

Summary Play a part in the next revolution in human-computer interaction and redefine mobile computing through voice interaction!

Apple's Siri AI Quality team is looking for a passionate and skilled AI Software Evaluation Engineer to join our mission of delivering the highest-quality AI experiences to millions of users worldwide. We own the end-to-end Siri pre-ship evaluation dataset and the synthetic personas that power it. In this role, you will combine agentic engineering, data analytics and AI evaluation experience to help us catch quality issues early, automate repetitive eval operations, and expand coverage as Siri's features and platforms grow.

Description In this role, you will build the systems that evaluate Siri's AI experiences and turn the results into action. Your work will span multiple areas of focus:

• Evaluation data insights. Investigate why test scenarios regress and which issues recur. Turn failure and bug data into root-cause findings and recommendations that product, engineering and partner teams act on.
• Coverage analysis. Find gaps across app intents, features and product areas, and partner with domain owners to close them. Build and report on dataset quality, issue trends and coverage.
• Agentic scenario generation, triage and auto-fix. Design agentic workflows that propose, ground, validate, triage and nominate new evaluation scenarios with minimal manual effort. Extend automated screening, triage and fix tooling to new domains and device platforms.

You will work with engineering, product, evaluator, persona and data-platform teams to define quality standards and deliver evaluation systems that guide Siri's development.

If you are a self-motivated engineer with a strong analytical mind, a bias toward automation and a desire to improve a product used by millions, we encourage you to apply.

Minimum Qualifications 5+ years of industry experience in software engineering, quality engineering, data engineering or ML engineering. 2+ years of hands-on experience building LLM-based applications or agents, including tool use, multi-step workflows and prompt engineering. Experience designing and running AI or LLM evaluation: building evaluation datasets, defining metrics and rubrics, LLM-as-judge or automated evaluators, and analyzing failures. Strong data analytics skills, including SQL and Python, and experience turning large datasets into clear insights and dashboards. Strong programming skills in Python and experience with automation and CI/CD. Strong communication and collaboration skills, with the ability to influence cross-functional partners and work across teams.

Preferred Qualifications Experience with agent frameworks and developer tooling, such as MCP servers, skills or plugins, and agentic testing. Experience with voice assistants, conversational AI or multi-turn dialogue evaluation. Experience with data schemas, validation frameworks, and structured data formats (e.g. JSON). Experience with data visualization and BI tools (e.g. Tableau) and with data warehouses (e.g. Snowflake). Familiarity with coverage analysis, traffic or taxonomy analysis, and root-cause analysis of flaky or regressing tests. Comfortable working in a fast-paced, collaborative environment with shifting priorities.

Pay & Benefits At Apple, base pay is one part of our total compensation package and is determined within a range. This provides the opportunity to progress as you grow and develop within a role. The base pay range for this role is between $150,400 and $277,600, and your base pay will depend on your skills, qualifications, experience, and location.

Apple employees also have the opportunity to become an Apple shareholder through participation in Apple’s discretionary employee stock programs. Apple employees are eligible for discretionary restricted stock unit awards, and can purchase Apple stock at a discount if voluntarily participating in Apple’s Employee Stock Purchase Plan. You’ll also receive benefits including: Comprehensive medical and dental coverage, retirement benefits, a range of discounted products and free services, and for formal education related to advancing your career at Apple, reimbursement for certain educational expenses - including tuition. Additionally, this role might be eligible for discretionary bonuses or commission payments as well as relocation. Learn more about Apple Benefits

Note: Apple benefit, compensation and employee stock programs are subject to eligibility requirements and other terms of the applicable plan or program.

EEO Statement Apple is an equal opportunity employer that is committed to inclusion and diversity. We seek to promote equal opportunity for all applicants without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, Veteran status, or other legally protected characteristics. Learn more about your EEO rights as an applicant

Accessibility At Apple, we believe accessibility is a fundamental human right. You’ll find that idea reflected in everything here - in our culture, our benefits and our digital tools. By welcoming as many perspectives as possible, we help you build a career where you feel like you belong.

Learn about accessibility in Apple’s workplace

Learn about reasonable accommodations for job applicants

Application Deadline Apple accepts applications to this posting on an ongoing basis.