Staff Engineer, Applied AI & Build Standards

As a Staff Engineer on Best Buy’s Applied AI & Build Standards team, you'll develop evaluation frameworks, quality standards and release controls that ensure agents perform safely and effectively. Working across the platform, you'll also help transform successful agent interactions into reusable capabilities and trusted organizational knowledge that accelerates future innovation. Our team, known internally as AI Rails, builds the shared platform and sets the build standards that let thousands of Best Buy engineers and technologists use AI agents with the right access, security and cost controls.

This role is hybrid, which means you will be required to work on site Monday through Thursday at the Best Buy corporate office in Richfield, Minnesota. You can work virtually from home or another non‑Best Buy location on Friday. The recruiter or hiring manager will provide more details during the hiring process.

This role is eligible for immigration sponsorship.

What you’ll do

• Own the roadmap and quality bar for agent evaluation across the platform, and influence architecture across teams
• Build eval datasets and harnesses from production agent traces, including trajectory evaluation for multi-step runs
• Wire evals into deployment pipelines as release gates, catching regressions as models and prompts change
• Calibrate LLM-as-judge scoring against human review, with annotation workflows that route the right traces to reviewers
• Run the learning loop: turn successful agent runs into reusable skills, with human-gated correction, replay and drift monitoring
• Own a permission-aware knowledge layer: source ingestion with provenance and freshness, and the context API that serves agents
• Use of Agentic AI as core development/coding tool

Basic qualifications

• 5 or more years of relevant experience with a Bachelor's degree OR equivalent relevant professional experience
• 3 or more years of professional software development experience in Python
• 2 or more years of experience building production data pipelines or search and retrieval systems, including data quality checks
• 1 or more years of experience with formal or informal leadership
• Hands-on experience building evaluation frameworks for LLM-based applications or AI agents
• Daily, hands-on use of AI coding agents in how you write, test and review code

Preferred qualifications

• Bachelor's degree in IT, Computer Science, Engineering, or related field
• 7 or more years of relevant professional experience
• Running eval suites as release gates in CI/CD pipelines
• Trajectory evaluation or simulation harnesses for multi-step, tool-using agents
• Calibrating LLM-as-judge evaluators against human labels and measuring their agreement
• Permission-aware retrieval or context systems that track source provenance and freshness
• Background in search relevance, recommendations or ML evaluation, including offline and online metrics

What’s in it for you We’re committed to helping our people thrive at work and at home. Best Buy offers a range of benefits to support your overall well-being and provide support as you need it, especially at key moments in your life.

Our benefits include:

• Competitive pay
• Great employee discount
• Financial savings and retirement resources
• Support for your physical and mental well-being

Best Buy provides different types of leaves of absence (LOA) and potential pay sources to employees based on eligibility. The length of your LOA depends on your situation, where you live, your full-time or part-time employment status, and federal and state regulations. Intermittent or reduced-schedule leave is also available for certain medical or family care leaves. Paid time off (vacation or PTO) is offered to full-time and part-time employees based on work location, employment status, salary or hourly status (exempt/non-exempt), and years of continued or bridged service.

Certain roles, where market norms demand it, are eligible for various forms of incentive pay to drive performance and offer recognition for achieving financial and strategic results. For more information about our incentive pay plans, including eligibility, please refer to our Incentive Programs Summary.

For more information about benefits, LOA and paid time off, please refer to our Benefits Guide.

About us

As part of the Best Buy team, you’ll help us fulfill our purpose to enrich lives through technology. We bring that to life every day by humanizing and personalizing tech solutions for every stage of life - in our stores, online and in customers’ homes.

Our culture is built on deeply supporting and valuing our amazing employees who make it all possible. We’re committed to being a great place to work, where you can unlock unique career possibilities. Above all, we aim to provide a place where you can bring your full, authentic self to work now and into the future. Tomorrow works here.™

Best Buy is an equal opportunity employer.

Application deadline: Minimum of 5 days from the posting date. You can find that date above the job title at the top of the page.