Jobs in the United States

25+ jobs

Save this search

Added 6 mins ago
Functional Service Team Executive/Senior Director, Clinical Delivery Capabilities – Exploratory Medicine Pharmacology (EMP)
Lilly
Indianapolis, IN
Ensure that the infrastructure required to support the service team is in place, including systems access, tools, data and reporting capabilities, contracting and budgeting mechanisms, and…
Added 6 mins ago
LillyDirect Pharmacy Master Data Technical Lead
Lilly
Indianapolis, IN
Set standards for CI/CD and infrastructure-as-code for data workloads, working with the DevOps engineer, so environments are repeatable and reviewed.
Added 6 mins ago
LillyDirect Pharmacy DevOps Engineer
Lilly
Indianapolis, IN
Develop and maintain infrastructure as code (Terraform, CloudFormation, or CDK) for AWS and Databricks environments, so they are repeatable and reviewed.
Added 6 mins ago
LillyDirect Pharmacy Data Engineer
Lilly
Indianapolis, IN
…building, and maintaining robust and scalable data pipelines and infrastructure to support our critical scientific and business initiatives, ultimately contributing to the delivery of…
Added 12 mins ago
Principal Counsel, Product & Commercial
Pax8, Inc
Remote, United States
Help build scalable Legal infrastructure, developing playbooks, templates, decision frameworks and self-service approaches that empower teams to move confidently.
Added 22 mins ago
Staff Machine Learning Engineer - Edge AI
Samsara
Remote, United States
…Representing more than 40% of global GDP, these industries are the infrastructure of our planet, including agriculture, construction, field services, transportation, and manufacturing - and…
Added 22 mins ago
Manager, Operations Program Management
Samsara
Remote, United States
…Representing more than 40% of global GDP, these industries are the infrastructure of our planet, including agriculture, construction, field services, transportation, and manufacturing - and…
Added 23 mins ago
Operations Data Engineering Analyst
Southern Company
Birmingham, AL
The Operations Data Engineering Analyst supports both operational data infrastructure and AI/ML data readiness. The role assists the PI System Administrator with daily operation,…
Added 23 mins ago
Business Development Director - Directed Energy Systems, International
AeroVironment
Huntsville, AL
…can support layered air defense, protect forces and critical infrastructure, and address the growing threat posed by unmanned aircraft…
Added 23 mins ago
Technical Project Manager
Leidos
Huntsville, AL
Coordinate program resources and infrastructure necessary for successful execution, including laboratory and office facilities, IT resources, Government Furnished Equipment (GFE), material…
Added 23 mins ago
Cyber Security Manager - Diego Garcia
Amentum
Montgomery, AL
…professional services serving the US government, allies and critical infrastructure sectors, is seeking an FRCS Cybersecurity Manager to support a government contract based on the island of…
Added 23 mins ago
Senior Manufacturing Engineer, Welding and Automation - Leeds, AL
Hubbell Incorporated
Leeds, AL
Hubbell creates critical infrastructure solutions that power our customers, communities, people and the planet. Our company is strategically aligned around enabling grid modernization and…
Added 23 mins ago
Quality Technician II - Leeds, AL
Hubbell Incorporated
Leeds, AL
Hubbell creates critical infrastructure solutions that power our customers, communities, people and the planet. Our company is strategically aligned around enabling grid modernization and…
Added 27 mins ago
GDOT Project Manager, PQM Coordinator 2
Atlas
Duluth, GA
…geotechnical consulting services for environmental, industrial and infrastructure construction projects. Headquartered in Austin, TX, Atlas currently has over 3,500 employees with offices…
Added 27 mins ago
GDOT Project Manager, PQM Coordinator 3
Atlas
Duluth, GA
…geotechnical consulting services for environmental, industrial and infrastructure construction projects. Headquartered in Austin, TX, Atlas currently has over 3,500 employees with offices…
Added 27 mins ago
CADD Manager/Senior Designer
Atlas
Plymouth Meeting, PA · PA
…geotechnical consulting services for environmental, industrial and infrastructure construction projects. Headquartered in Denver, CO, Atlas currently has over 3,500 employees with offices…
Added 27 mins ago
A
Environmental Project Manager, MCP (Massachusetts Contingency Plan)
Atlas
Woburn, MA · MA · Manchester, NH · NH
- $80,000 to $105,000 USD / year
…geotechnical consulting services for environmental, industrial and infrastructure construction projects. Headquartered in Denver, CO, Atlas currently has over 3,500 employees with offices…
Added 27 mins ago
Environmental Scientist/Biologist
Atlas
Tempe, AZ
Atlas is a national leader in environmental, industrial, and infrastructure consulting, providing the expertise and resources needed to move complex projects from concept to completion. Our…
Added 27 mins ago
Sr Highway Design Engineer/Project Manager
Atlas
Duluth, GA
…geotechnical consulting services for environmental, industrial and infrastructure construction projects. Headquartered in Austin, TX, Atlas currently has over 3,500 employees with offices…
Added 27 mins ago
Senior Bridge Hydraulics & Water Resources Engineer (PE)
Atlas
Duluth, GA
…geotechnical consulting services for environmental, industrial and infrastructure construction projects. Headquartered in Austin, TX, Atlas currently has over 3,500 employees with offices…
Added 27 mins ago
Senior Transportation Project Manager, Illinois PE/SE
Atlas
Chicago, IL
…geotechnical consulting services for environmental, industrial and infrastructure construction projects. Headquartered in Denver, CO, Atlas currently has over 3,500 employees with offices…
Added 27 mins ago
A
Senior Transportation Project Manager (Roadway Design & Client Development)
Atlas
Chicago, IL
- $275,000 USD / year
…geotechnical consulting services for environmental, industrial and infrastructure construction projects. Headquartered in Denver, CO, Atlas currently has over 3,500 employees with offices…
Added 27 mins ago
A
Senior Geotechnical Engineer
Atlas
Burlington, NJ · Philadelphia, PA · Cherry Hill, NJ · Mount Laurel, NJ · Maple Shade, NJ
- $140,000 to $175,000 USD / year
Atlas is a trusted leader in environmental and infrastructure consulting services; we provide services that enable communities to flourish and thrive. With a team of 3,500 employees and…
Added 27 mins ago
A
Regional Infrastructure Market Director - Geotechnical, Environmental & Construction Engineering Services
Atlas
Burlington, NJ · Philadelphia, PA · Maple Shade, NJ · Cherry Hill, NJ · Mount Laurel, NJ
- $175,000 to $250,000 USD / year
Atlas is a trusted leader in environmental and infrastructure consulting services; we provide services that enable communities to flourish and thrive. With a team of 3,500 employees and…
Added 27 mins ago
Construction Materials Testing Technician
Atlas
Columbus, OH · New Albany, IN
Atlas is a trusted leader in environmental and infrastructure consulting services; we provide services that enable communities to flourish and thrive. With a team of 3,500 employees and…

Microsoft AI

AI Evaluation Systems Engineering Lead

Mountain View, CA, United States

$142,800 to $331,200 USD / year

Apply About employer

Applications open in a new tab.

Job details

Pay: $142,800 to $331,200 USD / year
Posting language: English
Posted: Sep 29, 2026

Original job description

Overview

Microsoft AI is looking for a hands-on engineering lead to build the evaluation infrastructure that helps researchers improve frontier models and make reliable release decisions. You will lead the Eval Systems team, building systems that turn model checkpoints into trustworthy results - quickly, reproducibly, and at scale.

The role combines deep technical work with team leadership. You will shape the architecture and roadmap, contribute to implementation, and hire and develop engineers. Working closely with research, training, and inference teams, you will make evaluation a dependable part of the model-development loop, from early experiments through release readiness.

We are looking for someone who enjoys difficult systems problems, makes thoughtful engineering tradeoffs, and builds teams grounded in clear ownership, collaboration, and technical excellence.

Responsibilities

Set technical direction. Own the team’s architecture, roadmap, and delivery, balancing immediate research needs with long-term reliability and scalability.
Build end-to-end evaluation systems. Develop infrastructure for benchmark configuration, model inference, grading, aggregation, and results publication across training checkpoints and deployed models.
Accelerate research iteration. Reduce evaluation turnaround time and improve throughput, scheduling, and GPU efficiency. Make it easier for researchers to integrate benchmarks, investigate results, and compare models.
Make results trustworthy. Establish reproducible configurations, traceable results, validation, and regression checks. Partner with evaluation owners to preserve correctness and comparability as models and infrastructure evolve.
Own operational excellence. Define and track reliability and performance objectives. Improve observability, failure recovery, and on-call practices, addressing recurring problems at their source.
Stay hands-on. Contribute code, lead design reviews, and help debug challenging problems across evaluation, inference, and distributed execution.
Build and develop the team. Hire and mentor engineers, establish clear responsibilities, and foster an inclusive environment where people can do their best work.
Lead across teams. Align research and engineering partners on priorities, dependencies, and tradeoffs, and communicate progress and risks clearly.

Required Qualifications

Bachelor’s Degree in Computer Science or a related technical discipline AND 6+ years of technical engineering experience with coding in languages including C, C++, C#, Java, JavaScript, or Python.
OR equivalent experience.

Preferred Qualifications

Bachelor's Degree in Computer Science or related technical field AND 10+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python
Substantial software engineering experience building and operating production distributed systems, infrastructure platforms, or ML systems.
Demonstrated technical leadership, including setting direction and delivering complex projects through a team.
Strong programming skills, including proficiency in Python and experience contributing to large, shared codebases.
Experience diagnosing distributed-system failures and improving reliability, performance, and resource efficiency.
Experience with production operations, including monitoring, incident response, and on-call practices.
Ability to translate evolving requirements into clear engineering priorities and work effectively with research and engineering partners.
A track record of mentoring engineers and strengthening team engineering practices.
Experience with LLM evaluation, benchmarking, model-based grading, or reinforcement learning infrastructure.
Experience with GPU-backed inference, distributed training, or large-scale compute workloads.
Familiarity with distributed execution frameworks and schedulers such as Ray, Kubernetes, or Slurm.
Experience improving experimental reproducibility, validating metrics, or investigating differences across evaluation and serving environments.
Experience hiring engineers, managing people, or growing a technical team.
A track record of simplifying complex infrastructure and reducing operational burden through automation and systematic reliability improvements.

Software Engineering IC5 - The typical base pay range for this role across the U.S. is USD $142,800 - $274,800 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $188,000 - $304,200 per year. Software Engineering IC6 - The typical base pay range for this role across the U.S. is USD $165,600 - $296,400 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $220,800 - $331,200 per year.

Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here: https://careers.microsoft.com/us/en/us-corporate-pay

This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.

Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.

Apply About employer

Report this job