Jobs in the United States

25+ jobs

Save this search

Added 1 min ago
Workforce Planning & Budget Manager
Stellantis
Auburn Hills, MI
Defining and implementing the outsourcing strategy for the global SWE (Software Engineering) organization
Added 3 mins ago
Senior Mechanical Engineer – Robotics
Premier Global Links
Watertown · Watertown, MA
Working in a fast-growing hardware engineering environment, you will collaborate with mechanical, electrical, software, and deployment teams to turn innovative robotic concepts into…
Added 4 mins ago
Solutions Engineer – State, Local, Educational (SLED) Campus Security
Skydio
San Mateo, CA
…expertise in artificial intelligence, best-in-class hardware and software product development, operational excellence, and customer obsession to empower a broader, more diverse audience of…
Added 4 mins ago
Solutions Engineer – State, Local, Educational (SLED) Corrections Security
Skydio
San Mateo, CA
…facilities, capture the customer feedback that guides product development, and work cross-functionally to prepare Customer Success Plans. Priority will be given to candidates who have…
Added 5 mins ago
EC
Manager
Eze Castle Software LLC
United States
- $178,131 to $190,000 USD / year
…responsibilities. 3 years of experience serving in a technical team lead (non-management) capacity. 5 years of experience with Delphi programming language to support legacy codebase. 8…
Added 5 mins ago
CI
Controls Engineer
Cintal, Inc.
Tucson, AZ
- $90,000 to $95,000 USD / year
…advanced technologies to reduce product costs, shorten design and development lead times, and accelerate the introduction of high-quality products to the…
Added 7 mins ago
Senior Account Executive, SLED - New York
Horizon3
Remote, United States
…security culture, cybersecurity skills shortage, and the long lead time and expense of hiring outside consultants. Collectively, we are a team of learn it alls, committed to a culture of…
Added 7 mins ago
H
Engineering Manager, Internal Attack
Horizon3
Remote, United States
- $238,000 to $280,000 USD / year
This is the opportunity to lead the team that innovates on and maintains NodeZero's core internal attack capabilities. Internal pentesting is NodeZero's flagship capability and a core part…
Added 7 mins ago
Senior Account Executive, SLED - Texas
Horizon3
Remote, United States
…security culture, cybersecurity skills shortage, and the long lead time and expense of hiring outside consultants. Collectively, we are a team of learn it alls, committed to a culture of…
Added 7 mins ago
Senior Account Executive, SLED - California
Horizon3
Remote, United States
…security culture, cybersecurity skills shortage, and the long lead time and expense of hiring outside consultants. Collectively, we are a team of learn it alls, committed to a culture of…
Added 7 mins ago
Senior Account Executive, SLED - Illinois
Horizon3
Chicago, IL
…security culture, cybersecurity skills shortage, and the long lead time and expense of hiring outside consultants. Collectively, we are a team of learn it alls, committed to a culture of…
Added 7 mins ago
Senior Account Executive, SLED - Florida
Horizon3
Remote, United States
…security culture, cybersecurity skills shortage, and the long lead time and expense of hiring outside consultants. Collectively, we are a team of learn it alls, committed to a culture of…
Added 7 mins ago
MA
AI Evaluation Systems Engineering Lead
Microsoft AI
Mountain View, CA
- $142,800 to $331,200 USD / year
Microsoft AI is looking for a hands-on engineering lead to build the evaluation infrastructure that helps researchers improve frontier models and make reliable release decisions. You will…
Added 8 mins ago
Senior Product Manager, Pixel New Product Introduction
Google
Mountain View, CA
Collaborate deeply across Platform, Silicon, Tech Engineering and Software to land compelling prototypes that capture the imagination of the organization.
Added 8 mins ago
Software Engineer III, Infrastructure, Google Cloud Security and Privacy
Google
Sunnyvale, CA
Proficiency in code and system health, diagnosis and resolution, and software test engineering.
Added 9 mins ago
Senior Software Engineer, VMS
Anduril Industries
Quincy, MA
[Maritime & Manoeuvre Dominance : Undersea Reconnaissance & Strike - Engineering & Operations : Software
Added 9 mins ago
Technical Project Manager
Dynetics, Inc
Huntsville, AL
Lead cross-functional teams spanning software engineering, systems engineering, cybersecurity, Information Assurance, IT, test, security, facilities, finance, contracts, procurement, and…
Added 9 mins ago
Power Delivery Project Manager
LEIDOS ENGINEERING INC
Walled Lake, MI
Leidos Engineering has an opportunity for a Project Manager I (PM I) who will work alongside lead engineers and account managers supporting substation engineering projects for various…
Added 9 mins ago
Senior Program Director
LEIDOS, INC.
Reston, VA
·Minimum of 10 years of experience managing complex programs in IT, cloud, communications, software development, network, infrastructure, or a related discipline.
Added 9 mins ago
RF Communications Engineer
Dynetics, Inc
Huntsville, AL
The Lead RF Communications Engineer will support the Leidos Space Platforms Portfolio in the development of a lunar terrain vehicle system and support equipment, primarily responsible for…
Added 9 mins ago
Senior Industrial Engineer
The Productivity Team
United States
Lead manufacturing and industrial engineering initiatives supporting production startup, optimization, and operational excellence.
Added 10 mins ago
ND
Principal AI & ML Architect
National Debt Relief, LLC.
Remote, United States
- $180,000 to $207,000 USD / year
8+ years of experience in ML engineering, data science, AI engineering, or software architecture roles.
Added 10 mins ago
Engineering Operations Lead
Venturi
Huntsville, AL
Venturi, LLC A Chenega Company has an exciting career opportunity for an experienced Engineering Operations Lead to support the Missile Defense Agency (MDA) Targets and Countermeasures…
Added 10 mins ago
Engineering Designer III- Civil/Site
Colliers Engineering & Design
McLean, VA
This is a great opportunity for an experienced civil engineering professional who is ready to take on more responsibility and play an active role in a variety of land development projects.…
Added 11 mins ago
Technical Services Manager
Tyson Shared Services, Inc.
Council Bluffs, IA
1ST SHIFT (United States of America)This role provides leadership and technical direction for Research and Development Technical Services at assigned manufacturing facilities and…

Microsoft AI

AI Evaluation Systems Engineering Lead

Mountain View, CA, United States

$142,800 to $331,200 USD / year

Apply About employer

Applications open in a new tab.

Job details

Pay: $142,800 to $331,200 USD / year
Posting language: English
Posted: Sep 29, 2026

Original job description

Overview

Microsoft AI is looking for a hands-on engineering lead to build the evaluation infrastructure that helps researchers improve frontier models and make reliable release decisions. You will lead the Eval Systems team, building systems that turn model checkpoints into trustworthy results - quickly, reproducibly, and at scale.

The role combines deep technical work with team leadership. You will shape the architecture and roadmap, contribute to implementation, and hire and develop engineers. Working closely with research, training, and inference teams, you will make evaluation a dependable part of the model-development loop, from early experiments through release readiness.

We are looking for someone who enjoys difficult systems problems, makes thoughtful engineering tradeoffs, and builds teams grounded in clear ownership, collaboration, and technical excellence.

Responsibilities

Set technical direction. Own the team’s architecture, roadmap, and delivery, balancing immediate research needs with long-term reliability and scalability.
Build end-to-end evaluation systems. Develop infrastructure for benchmark configuration, model inference, grading, aggregation, and results publication across training checkpoints and deployed models.
Accelerate research iteration. Reduce evaluation turnaround time and improve throughput, scheduling, and GPU efficiency. Make it easier for researchers to integrate benchmarks, investigate results, and compare models.
Make results trustworthy. Establish reproducible configurations, traceable results, validation, and regression checks. Partner with evaluation owners to preserve correctness and comparability as models and infrastructure evolve.
Own operational excellence. Define and track reliability and performance objectives. Improve observability, failure recovery, and on-call practices, addressing recurring problems at their source.
Stay hands-on. Contribute code, lead design reviews, and help debug challenging problems across evaluation, inference, and distributed execution.
Build and develop the team. Hire and mentor engineers, establish clear responsibilities, and foster an inclusive environment where people can do their best work.
Lead across teams. Align research and engineering partners on priorities, dependencies, and tradeoffs, and communicate progress and risks clearly.

Required Qualifications

Bachelor’s Degree in Computer Science or a related technical discipline AND 6+ years of technical engineering experience with coding in languages including C, C++, C#, Java, JavaScript, or Python.
OR equivalent experience.

Preferred Qualifications

Bachelor's Degree in Computer Science or related technical field AND 10+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python
Substantial software engineering experience building and operating production distributed systems, infrastructure platforms, or ML systems.
Demonstrated technical leadership, including setting direction and delivering complex projects through a team.
Strong programming skills, including proficiency in Python and experience contributing to large, shared codebases.
Experience diagnosing distributed-system failures and improving reliability, performance, and resource efficiency.
Experience with production operations, including monitoring, incident response, and on-call practices.
Ability to translate evolving requirements into clear engineering priorities and work effectively with research and engineering partners.
A track record of mentoring engineers and strengthening team engineering practices.
Experience with LLM evaluation, benchmarking, model-based grading, or reinforcement learning infrastructure.
Experience with GPU-backed inference, distributed training, or large-scale compute workloads.
Familiarity with distributed execution frameworks and schedulers such as Ray, Kubernetes, or Slurm.
Experience improving experimental reproducibility, validating metrics, or investigating differences across evaluation and serving environments.
Experience hiring engineers, managing people, or growing a technical team.
A track record of simplifying complex infrastructure and reducing operational burden through automation and systematic reliability improvements.

Software Engineering IC5 - The typical base pay range for this role across the U.S. is USD $142,800 - $274,800 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $188,000 - $304,200 per year. Software Engineering IC6 - The typical base pay range for this role across the U.S. is USD $165,600 - $296,400 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $220,800 - $331,200 per year.

Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here: https://careers.microsoft.com/us/en/us-corporate-pay

This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.

Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.

Apply About employer

Report this job