Find work in the United States.

6,265,094 jobs across our US network. How we count

Jobs in the United States

25+ jobs

Save this search

Added just now
T-Ads Engineer, Software Full Stack
T-Mobile USA, Inc.
Bellevue, WA
…end services, microservices, databases, and associated cloud infrastructure. The role is distinguished by responsibility for end-to-end system implementation, deployment, maintenance, and…
Added just now
Senior Mechanical Engineer
Momentus Space LLC
San Jose, CA
As an early mover in the in-space transportation and infrastructure services, Momentus is at the forefront of the commercialization of space. With an experienced team of aerospace,…
Added just now
AI Ops Engineer (GTM) - Mid Level
Orbital
New York City, NY
Real estate shapes the world around us: our homes, our workplaces and our cultural landmarks. It’s the infrastructure that powers everyday life and is the world's largest asset class. Yet…
Added just now
AI Ops Engineer (GTM)
Orbital
New York City, NY
Real estate shapes the world around us: our homes, our workplaces and our cultural landmarks. It’s the infrastructure that powers everyday life and is the world's largest asset class. Yet…
Added 2 mins ago
A
Senior Software Engineer
Akamai
Remote, United States
- $121,400 to $218,600 USD / year
As a Software Engineer Senior, you will be responsible for:
Added 5 mins ago
UG
Principal Cybersecurity Analyst (Attack Surface Mgt.) On Site MN
UnitedHealth Group
United States
- $112,700 to $193,200 USD / year
Perform continuous discovery, validation, and risk analysis of public-facing infrastructure, cloud environments, domains, APIs, and third-party dependencies to pinpoint vulnerabilities and…
Added 5 mins ago
UG
Oracle Exadata DBA - Remote
UnitedHealth Group
NY
- $112,700 to $193,200 USD / year
Join the NY APD and Virginia EDW Analytics Operations and Maintenance team within Optum Technology as a Principal Data Engineer. In this role, you will lead the architecture, design, and…
Added 6 mins ago
IT Cloud Engineer, Engineering Platform Automation
Mayo Clinic
Rochester, MN · Rochester
The IT Cloud Engineer – Engineering Platform Automation is responsible for designing, developing, operating, and continuously evolving Mayo Clinic's Engineering Front Door - a cloud-native…
Added 6 mins ago
Senior Process Engineer
Salas O'Brien
Remote, United States
…building types as well as structural and building sciences, infrastructure asset management, advanced robotics, and…
Added 6 mins ago
Sales Engineer
GLOBO Language Solutions, LLC
Remote, United States
Primary Goal: As a Sales Engineer, you’ll be the technical expert in our sales process, partnering with Account Executives to help prospects understand how our solution solves their…
Added 7 mins ago
Associate Director, AI Engineering
CSL Behring L.L.C.
United States
…scientists. The Associate Director is the team's senior full-stack engineer and architect. The role owns the engineering standards, development and deployment patterns, and production…
Added 8 mins ago
Senior Attack Engineer, GCP
Horizon3
Remote, United States
We're looking for an Attack Engineer with deep Google Cloud Platform (GCP) expertise to help build the next generation of cloud attack capabilities within NodeZero.
Added 8 mins ago
Senior Attack Engineer, Kubernetes
Horizon3
Remote, United States
We’re looking for a senior offensive security engineer to lead the development of Kubernetes attack capabilities in NodeZero. You’ll research how attackers move from a compromised workload…
Added 8 mins ago
Senior Attack Engineer, OCI
Horizon3
Remote, United States
We're looking for an Attack Engineer with Oracle Cloud Infrastructure (OCI) expertise to help build the next generation of cloud attack capabilities within NodeZero.
Added 8 mins ago
Principal Database Platform Engineer
Boston Consulting Group
Boston, MA · Atlanta, GA
This role suits an engineer who has operated at a senior level before – someone who identifies what needs to be built, makes sound technical decisions, and delivers without requiring…
Added 8 mins ago
VS
Senior Systems Administrator (L3) – Azure Government & Windows Infrastructure
Virtual Service Operations
Reston, VA
- $115,000 to $135,000 USD / year
The ideal candidate is a highly technical, hands-on infrastructure engineer capable of independently deploying new environments, resolving escalated technical issues, and supporting secure,…
Added 9 mins ago
UG
Principal Cybersecurity Analyst (Attack Surface Mgt.) On Site MN
UnitedHealth Group
Columbia, MD
- $112,700 to $193,200 USD / year
Perform continuous discovery, validation, and risk analysis of public-facing infrastructure, cloud environments, domains, APIs, and third-party dependencies to pinpoint vulnerabilities and…
Added 11 mins ago
IT Cloud Engineer, Engineering Platform Automation
Mayo US
Rochester, MN
The IT Cloud Engineer – Engineering Platform Automation is responsible for designing, developing, operating, and continuously evolving Mayo Clinic's Engineering Front Door - a cloud-native…
Added 11 mins ago
GI
Forward Deployed Engineer
Globality, Inc.
Palo Alto, CA
- $100,000 to $200,000 USD / year
As a Forward Deployed Engineer at Globality, you will help enterprise procurement teams realize the full value of Glo by running it inside their own systems and aligning it to their own…
Added 11 mins ago
SAP NS2 Senior Cloud Security Engineer-TS/SCI + poly Onsite - Chantilly, VA
SAP
Herndon, VA
…meet the requirements of U.S. national security and critical infrastructure…
Added 13 mins ago
A
ITS/Traffic Operations Engineer
AECOM
Mount Clemens, MI
- $47,125 to $61,625 USD / year
…helps people and communities thrive. We are the world's trusted infrastructure consulting firm, partnering with clients to solve the world’s most complex challenges and build legacies for…
Added 13 mins ago
A
Transit Market Sector Leader
AECOM
Greenville, SC
- $180,000 to $248,000 USD / year
…helps people and communities thrive. We are the world's trusted infrastructure consulting firm, partnering with clients to solve the worldâs most complex challenges and build legacies for…
Added 13 mins ago
A
Transit Market Sector Leader
AECOM
Charlotte, NC
- $180,000 to $248,000 USD / year
…helps people and communities thrive. We are the world's trusted infrastructure consulting firm, partnering with clients to solve the worldâs most complex challenges and build legacies for…
Added 13 mins ago
A
Transit Market Sector Leader
AECOM
Columbia, SC
- $180,000 to $248,000 USD / year
…helps people and communities thrive. We are the world's trusted infrastructure consulting firm, partnering with clients to solve the worldâs most complex challenges and build legacies for…
Added 13 mins ago
A
Transit Market Sector Leader
AECOM
Birmingham, AL
- $180,000 to $248,000 USD / year
…helps people and communities thrive. We are the world's trusted infrastructure consulting firm, partnering with clients to solve the worldâs most complex challenges and build legacies for…

TrueFoundry

Senior AI/ML Engineer - LLM & Agent Stack (Customer Facing)

NY, United States
San Mateo, CA, United States

Apply About employer

Applications open in a new tab.

Job details

Experience: 5–10 years of software engineering with substantial experience building distributed systems, infra, or ML platforms.
Posting language: English
Posted: Nov 6, 2026

Original job description

About TrueFoundry

Every production AI system, whether it's powering customer support, writing code, analysing financial data, or diagnosing medical conditions, needs the same foundational infrastructure.A way to route between models. A way to manage tools and integrate them securely. A way to orchestrate agents and enforce governance. A unified compute layer to run it all.

That infrastructure layer is being built right now.

We are looking for a Senior AI/ML Engineer - LLM & Agent Stack (Customer-Facing) to join the team.

The Problem We're Solving

Companies are moving beyond simple chatbots to production agentic systems. These systems route between OpenAI, Anthropic, Google, and self-hosted models. They integrate dozens of tools via protocols like MCP. They orchestrate multi-agent workflows where agents coordinate with other agents.

The infrastructure to support this doesn't exist yet. You can't just duct-tape together a few API calls and call it production-ready.

You need a control plane that handles:

Intelligent routing with observability, cost policies, and fallback logic
Centralized tool and MCP server management with security and lifecycle controls
Agent orchestration with governance and guardrails
A unified compute layer to run self-hosted models, custom tools, and agents

AI Gateway is the control plane: five composable components (Prompts, LLM Gateway, MCP Gateway, Guardrails, Agent Gateway) that handle routing, orchestration, and governance.

We're Series A, backed by Intel Capital and Sequoia. Companies like CVS, Mastercard, Siemens, Paytm, Synopsys, and Zscaler run production AI workloads on our platform.

Role summary

You’ll design and own core components that enable enterprise customers to run production agentic AI safely and efficiently on TrueFoundry. This includes building robust orchestration for multi-step agents (graph/stateful workflows), model/routing logic, observability and policy enforcement (cost, data residency, rate limiting), and integrating upstream tooling like LangGraph, LangChain, vector stores, and specialized LLM runtimes. This is a customer-facing role where you'll work closely with TrueFoundry's enterprise customers. You may also need to write code in their environments.

What you’ll do

• Architect and implement scalable agent orchestration patterns (graph-based executors, state management, multi-agent coordination) for production workloads.

• Own critical integrations: model adapters, LLM gateway hooks, vector DBs, tools & external APIs, and the platform’s LLMops flows.

• Build and improve tracing, benchmarking and observability for LLMs and agents - token/cost accounting, latency p95, throughput, and correctness checks.

• Drive design for safety/guardrails: moderation hooks, human-in-the-loop checkpoints, replayable audit trails and policy enforcement.

• Mentor junior engineers, run design reviews, and improve engineering practices (testing, CI/CD, chaos testing for agents).

• Work directly with strategic customers to prototype complex agentic solutions and translate them into product features.

Must-have

• 5–10 years of software engineering with substantial experience building distributed systems, infra, or ML platforms.

• Deep practical experience integrating and deploying LLMs in production (RAG, retrieval, embeddings pipelines).

• Hands-on experience with agent orchestration frameworks (LangGraph / LangChain or custom agent runtimes) and stateful workflow design.

• Strong systems knowledge: Kubernetes, container orchestration, service meshes, and performance tuning.

• Proven track record building observability, cost controls, and policy enforcement for production services.

Preferred / differentiators

• Experience building or contributing to open-source LLM orchestration tools (LangGraph, LangChain, or similar).

• Familiarity with enterprise constraints: on-prem/cloud hybrid deployments, data residency, compliance requirements.

• Background in security, privacy, or model governance for LLMs.

• Demonstrated leadership in cross-functional projects and direct customer engagement.

Traits we are looking for: ownership, ability to execute, hustle and thinking out of the box, data-driven decision making, be comfortable with more unknowns than knowns.

Perks of Working at TrueFoundry

Join a fast-growing Series A, Bay Area-based startup building cutting-edge AI infrastructure.
Comprehensive health insurance for you and your family, including medical, dental, and vision coverage.
401(k) retirement plan.
Flexible hybrid work, 2 days a week in the office (Tuesday & Wednesday), with flexibility around the schedule.
Lunch and snacks are on us whenever you come into the office.
Work from another TrueFoundry office for up to 1 month each year, whether that's our London or India office, if you'd like a change of scenery.

Apply About employer

More about this job

Skills

ML
LangChain
Cost accounting
CI/CD
Workflow
Kubernetes
Regulatory compliance
Leadership
Decision making

Report this job