Jobs in the United States

25+ jobs

Save this search

Added just now
Help Desk Agent - Level 1
SECURITYINSPECTION.COM, INC DBA SOLVIT, INC.
Rochester Hills, MI
SolvIT, Inc. provides a strategic approach to technology, leveraging cost-effective business automation solutions for our customers. Our solutions include software development,…
Added just now
Sales & Business Development Associate
CCH Incorporated
Kennesaw, GA · Coppell, TX · Riverwoods, IL · Chicago, IL · Draper, UT
Basic comfort with AI-assisted prospecting tools, analytics platforms, or automation workflows
Added 1 min ago
Category Lead, Capital Equipment (CAPEX)
Ciena Canada ULC
Remote, United States
…investments may include test and measurement equipment, manufacturing automation, laboratory infrastructure, reliability testing systems, production tooling, and other technology-enabling…
Added 1 min ago
HK
Compliance & Enrollment Operations Manager
HEALTHY KIDS PROGRAMS
Danbury, CT
- $55,000 to $65,000 USD / year
We're also investing in automation to reduce repetitive work and improve efficiency. You'll help identify opportunities, work with each team's designated automation point person, coordinate…
Added 1 min ago
HK
Customer Experience & Automation Specialist
HEALTHY KIDS PROGRAMS
Danbury, CT
- $18 to $21 USD / hour
Healthy Kids Programs is seeking a Customer Experience & Automation Specialist to support families through the enrollment process while helping us improve and streamline the systems behind…
Added 1 min ago
HK
Customer Experience & Automation Manager
HEALTHY KIDS PROGRAMS
Danbury, CT
- $24 to $26 USD / hour
Healthy Kids Programs is seeking a Customer Experience & Automation Manager to support families through the enrollment process while helping us improve and streamline the systems behind it.
Added 3 mins ago
Service Desk Manager
Metro Cruise Services, LLC
Long Beach, CA
Added 3 mins ago
Operational Excellence Project Manager
Sentry Management
Longwood, FL
Identify opportunities to use technology and automation to meet business needs.
Added 3 mins ago
HRIS & Business Analyst
SFBLI
Jackson, MS
Lead HR technology projects and process improvement and automation initiatives.
Added 3 mins ago
Site Reliability Engineer (Infrastructure, Virtualization, & Compute)
SpaceX
Memphis, TN
Experience deploying and supporting Linux servers in physical and virtualized environments (e.g. VMware via automation).
Added 4 mins ago
Senior Mechanical Engineer – Robotics
Premier Global Links
Watertown · Watertown, MA
Experience developing robotic systems, industrial automation equipment, or advanced hardware products.
Added 4 mins ago
PG
Senior Electrical Engineer – Robotics
Premier Global Links
Watertown · Watertown, MA
- $150,000 to $220,000 USD / year
Experience developing electrical systems for robotics or industrial automation.
Added 4 mins ago
MC
Data Analyst
Miller Cooper & Co., Ltd.
Chicago, IL
- $65,000 to $70,000 USD / year
Interest in AI, automation, and advanced analytics.
Added 4 mins ago
Service Desk Manager
Metro Shore Services, LLC
Long Beach, CA
Added 4 mins ago
Security Researcher
Iru
United States
…identity & access, endpoint security & management, and compliance automation - collapsing the stack and giving IT & security time and control…
Added 4 mins ago
Principal Product Manager, Data
Kiavi
CA or Pittsburgh, PA
Added 4 mins ago
DC Trainer Shift 1
At Home
Carlisle, PA
Added 4 mins ago
SWING SHIFT ELECTRICAL & INSTRUMENTATION TECHNICIAN
Fuji Vegetable Oil USA
Savannah, GA
Diagnose PLC-controlled equipment, control circuits, automation and industrial network issues within authorized scope.
Added 5 mins ago
EC
Manager
Eze Castle Software LLC
United States
- $178,131 to $190,000 USD / year
…such as WinForms, WPF, DevExpress. 8 years of experience with automation testing frameworks like MSTest, TestComplete, Moq. 8 years of experience with troubleshooting tools like WindDbg,…
Added 6 mins ago
UX Developer
System One
Pittsburgh, PA
Develop AI-powered agents and copilots, including solutions that leverage SharePoint content for knowledge retrieval, search, and workflow automation.
Added 6 mins ago
PT
Full Stack Developer I
Paul Transportation Inc
Tulsa, OK
- $55,000 to $60,000 USD / year
As a Full-Stack Developer, you will work closely with IT leadership and operational teams to build and maintain applications that support daily business functions. This includes developing…
Added 6 mins ago
CI
Controls Engineer
Cintal, Inc.
Tucson, AZ
- $90,000 to $95,000 USD / year
The Automation Validation Engineer position develops, tests, and troubleshoots autonomy software systems used for autonomous machines. This generally includes onboard system validation work…
Added 6 mins ago
Senior Valuation Analyst (Real Estate)
Albertsons Companies
BOISE, ID · Boise, ID
Advance process improvement through standardized data, scalable models, automation, advanced analytics, and responsible use of AI-enabled tools.
Added 6 mins ago
National Category Director - Fresh Meat
Albertsons Companies
PLEASANTON, CA · BOISE, ID · Pleasanton, CA · Boise, ID
Added 7 mins ago
MA
AI Evaluation Systems Engineering Lead
Microsoft AI
Mountain View, CA
- $142,800 to $331,200 USD / year
A track record of simplifying complex infrastructure and reducing operational burden through automation and systematic reliability improvements.

Microsoft AI

AI Evaluation Systems Engineering Lead

Mountain View, CA, United States

$142,800 to $331,200 USD / year

Apply About employer

Applications open in a new tab.

Job details

Pay: $142,800 to $331,200 USD / year
Posting language: English
Posted: Sep 29, 2026

Original job description

Overview

Microsoft AI is looking for a hands-on engineering lead to build the evaluation infrastructure that helps researchers improve frontier models and make reliable release decisions. You will lead the Eval Systems team, building systems that turn model checkpoints into trustworthy results - quickly, reproducibly, and at scale.

The role combines deep technical work with team leadership. You will shape the architecture and roadmap, contribute to implementation, and hire and develop engineers. Working closely with research, training, and inference teams, you will make evaluation a dependable part of the model-development loop, from early experiments through release readiness.

We are looking for someone who enjoys difficult systems problems, makes thoughtful engineering tradeoffs, and builds teams grounded in clear ownership, collaboration, and technical excellence.

Responsibilities

Set technical direction. Own the team’s architecture, roadmap, and delivery, balancing immediate research needs with long-term reliability and scalability.
Build end-to-end evaluation systems. Develop infrastructure for benchmark configuration, model inference, grading, aggregation, and results publication across training checkpoints and deployed models.
Accelerate research iteration. Reduce evaluation turnaround time and improve throughput, scheduling, and GPU efficiency. Make it easier for researchers to integrate benchmarks, investigate results, and compare models.
Make results trustworthy. Establish reproducible configurations, traceable results, validation, and regression checks. Partner with evaluation owners to preserve correctness and comparability as models and infrastructure evolve.
Own operational excellence. Define and track reliability and performance objectives. Improve observability, failure recovery, and on-call practices, addressing recurring problems at their source.
Stay hands-on. Contribute code, lead design reviews, and help debug challenging problems across evaluation, inference, and distributed execution.
Build and develop the team. Hire and mentor engineers, establish clear responsibilities, and foster an inclusive environment where people can do their best work.
Lead across teams. Align research and engineering partners on priorities, dependencies, and tradeoffs, and communicate progress and risks clearly.

Required Qualifications

Bachelor’s Degree in Computer Science or a related technical discipline AND 6+ years of technical engineering experience with coding in languages including C, C++, C#, Java, JavaScript, or Python.
OR equivalent experience.

Preferred Qualifications

Bachelor's Degree in Computer Science or related technical field AND 10+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python
Substantial software engineering experience building and operating production distributed systems, infrastructure platforms, or ML systems.
Demonstrated technical leadership, including setting direction and delivering complex projects through a team.
Strong programming skills, including proficiency in Python and experience contributing to large, shared codebases.
Experience diagnosing distributed-system failures and improving reliability, performance, and resource efficiency.
Experience with production operations, including monitoring, incident response, and on-call practices.
Ability to translate evolving requirements into clear engineering priorities and work effectively with research and engineering partners.
A track record of mentoring engineers and strengthening team engineering practices.
Experience with LLM evaluation, benchmarking, model-based grading, or reinforcement learning infrastructure.
Experience with GPU-backed inference, distributed training, or large-scale compute workloads.
Familiarity with distributed execution frameworks and schedulers such as Ray, Kubernetes, or Slurm.
Experience improving experimental reproducibility, validating metrics, or investigating differences across evaluation and serving environments.
Experience hiring engineers, managing people, or growing a technical team.
A track record of simplifying complex infrastructure and reducing operational burden through automation and systematic reliability improvements.

Software Engineering IC5 - The typical base pay range for this role across the U.S. is USD $142,800 - $274,800 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $188,000 - $304,200 per year. Software Engineering IC6 - The typical base pay range for this role across the U.S. is USD $165,600 - $296,400 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $220,800 - $331,200 per year.

Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here: https://careers.microsoft.com/us/en/us-corporate-pay

This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.

Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.

Apply About employer

Report this job