Find work in the United States.

A simpler way to find your next job.

Jobs in the United States

25+ jobs

Save this search

Added 8 mins ago
Linux Software Engineer
Kastel Staffing
Huntsville, AL
We are seeking a UAS Linux Software Engineer to develop, integrate, and maintain the core software components that run on our drone platforms. This role focuses on the Linux‑based onboard…
Added 15 mins ago
Software System Engineer
The Exploration Company
Houston, TX
We are seeking a highly motivated and experienced Senior Software System Engineer to support the certification activities of the flight software for the Nyx program. You will work in a…
Added 16 mins ago
TT
Founding Forward Deployed Engineer
True Talent Solutions Partners
San Francisco, CA
- $180,000 to $230,000 USD / year
Join an early-stage AI customer success software team as a Founding Forward Deployed Engineer. You will work directly with enterprise customers to put AI agents into production, deliver…
Added 17 mins ago
K
Sr. Al Solution Engineer
Kentro
UNAVAILABLE
- $101,000 to $140,000 USD / year
Kentro is hiring for a Sr. AI Solution Engineer to support secure cloud data and applied AI initiatives for government customers. You will build governed data pipelines, automate cloud and…
Added 21 mins ago
SH
Business Intelligence Engineer III, Workday and AI Copilot
Sutter Health
Remote, United States
- $93,953.6 to $176,155.2 USD / year
A specialized mid-level technical professional who exercises independent judgment to architect, engineer, and optimize full-stack data products. This role acts as a technical authority,…
Added 26 mins ago
Sr. Engineer Firmware Engineering XII
Vertiv
United States
The Sr. Firmware Engineer will develop, sustain, and provide technical ownership of embedded firmware solutions for Single Phase UPS products. The role focuses on real-time DSP/MCU firmware…
Added 27 mins ago
RH
Software Engineer
Robert Half
Remote, United States
- $55 to $63 USD / hour
…interoperability solutions. We are seeking a Senior .NET Software Engineer with strong Microsoft technology expertise to design, build, and maintain scalable applications and integrations…
Added 29 mins ago
Software Integration Engineer, Autonomy
42dot
San Francisco
Added 29 mins ago
Sr. Software Integration Engineer, Autonomy
42dot
San Francisco
Added 29 mins ago
RH
DevOps II / SRE
Robert Half
Philadelphia, PA
- $40 to $45 USD / hour
We are seeking a DevOps II / Site Reliability Engineer to help build and mature a growing Site Reliability Engineering (SRE) function supporting large-scale digital platforms. This role…
Added 34 mins ago
Senior Member of Technical Staff, Platform Engineer (Infra)
Pallet
San Francisco, CA · NY
You could be an Infrastructure Engineer anywhere. Why us?
Added 34 mins ago
Senior Member of Technical Staff, Platform Engineer (Infra)
Pallet
NY · San Francisco, CA
You could be an Infrastructure Engineer anywhere. Why us?
Added 1 hour ago
Sr. DevOps Engineer
Forta Solutions, LLC
Denver, CO
We are looking for a strong Senior DevOps Engineer who can help shape the future of Warehouse Lending technology by working closely product leadership, engineering, QA, and implementation…
Added 1 hour ago
Infrastructure Engineer, Enterprise AI
Wonder Group, INC
Remote, United States
Wonder’s Enterprise Technology team is hiring an Infrastructure Engineer to establish and scale the hosting, deployment, and integration foundations that support builders of all technical…
Added 1 hour ago
SO
Software Engineer (Systems Programmer/Developer III) - 61161323
State of South Carolina
West Columbia, SC
- $67,728 to $121,890 USD / year
The South Carolina Department of Education is seeking a skilled Software Engineer to design, build, and support the applications and integrations that serve students, educators, and…
Added 1 hour ago
IAM Engineer
SpaceXSI
Austin, TX · Palo Alto, CA · Washington, DC
We are looking for an Identity and Access Management (IAM) Engineer to play a critical role in designing, implementing, and maturing our enterprise IAM ecosystem. As a Senior IAM Engineer,…
Added 1 hour ago
Principal Software Engineer - Simulation Data, Validation & Infrastructure
Irving, TX
…intelligent automation, and rigorous validation. This Principal Engineer role is a hands-on technical leadership position responsible for architecture, critical implementation, engineering…
Added 1 hour ago
Principal Software Engineer - Robotics & Physical AI Simulation
Irving, TX
This Principal Engineer role is a hands-on technical leadership position responsible for architecture, critical implementation, engineering standards, cross-team alignment, and mentoring.
Added 1 hour ago
Principal Software Engineer - Digital Twin and Simulation Platform
Irving, TX
This Principal Engineer role is a hands-on technical leadership position responsible for architecture, critical implementation, engineering standards, cross-team alignment, and mentoring.
Added 1 hour ago
Principal Software Engineer - Agentic AI & Simulation Automation
Irving, TX
This Principal Engineer role is a hands-on technical leadership position responsible for architecture, critical implementation, engineering standards, cross-team alignment, and mentoring.
Added 1 hour ago
Forward Deployed Solution Engineer – Applied AI FDE
ServiceNow
Remote, United States
It all started when engineer Fred Luddy wrote code that automated a tedious task for his coworker, Phyllis. She cried tears of joy. That moment inspired Fred to build a company that could…
Added 1 hour ago
Software engineer
SEI INVESTMENTS COMPANY
Oaks, PA
Added 1 hour ago
DevOps II / SRE
Robert Half
Philadelphia, PA
We are seeking a DevOps II / Site Reliability Engineer to help build and mature a growing Site Reliability Engineering (SRE) function supporting large-scale digital platforms. This role…
Added 1 hour ago
Regional Maintenance Manager
Amazon.com Services LLC
Kearny, NJ · Jessup, PA
Added 1 hour ago
Systems Integration/Test Engineer Stf (Level IV)
Lockheed Martin
King of Prussia, PA
…searching for a motivated and passionate Mission Operations Support Engineer Lead to join our Ground Product Lines Mission Support Team in King of Prussia, PA or Boulder, CO . In this role,…

TrueFoundry

Senior AI/ML Engineer - LLM & Agent Stack (Customer Facing)

NY, United States
San Mateo, CA, United States

Apply About employer

Applications open in a new tab.

Job details

Experience: 5–10 years of software engineering with substantial experience building distributed systems, infra, or ML platforms.
Posting language: English
Posted: Nov 6, 2026

Original job description

About TrueFoundry

Every production AI system, whether it's powering customer support, writing code, analysing financial data, or diagnosing medical conditions, needs the same foundational infrastructure.A way to route between models. A way to manage tools and integrate them securely. A way to orchestrate agents and enforce governance. A unified compute layer to run it all.

That infrastructure layer is being built right now.

We are looking for a Senior AI/ML Engineer - LLM & Agent Stack (Customer-Facing) to join the team.

The Problem We're Solving

Companies are moving beyond simple chatbots to production agentic systems. These systems route between OpenAI, Anthropic, Google, and self-hosted models. They integrate dozens of tools via protocols like MCP. They orchestrate multi-agent workflows where agents coordinate with other agents.

The infrastructure to support this doesn't exist yet. You can't just duct-tape together a few API calls and call it production-ready.

You need a control plane that handles:

Intelligent routing with observability, cost policies, and fallback logic
Centralized tool and MCP server management with security and lifecycle controls
Agent orchestration with governance and guardrails
A unified compute layer to run self-hosted models, custom tools, and agents

AI Gateway is the control plane: five composable components (Prompts, LLM Gateway, MCP Gateway, Guardrails, Agent Gateway) that handle routing, orchestration, and governance.

We're Series A, backed by Intel Capital and Sequoia. Companies like CVS, Mastercard, Siemens, Paytm, Synopsys, and Zscaler run production AI workloads on our platform.

Role summary

You’ll design and own core components that enable enterprise customers to run production agentic AI safely and efficiently on TrueFoundry. This includes building robust orchestration for multi-step agents (graph/stateful workflows), model/routing logic, observability and policy enforcement (cost, data residency, rate limiting), and integrating upstream tooling like LangGraph, LangChain, vector stores, and specialized LLM runtimes. This is a customer-facing role where you'll work closely with TrueFoundry's enterprise customers. You may also need to write code in their environments.

What you’ll do

• Architect and implement scalable agent orchestration patterns (graph-based executors, state management, multi-agent coordination) for production workloads.

• Own critical integrations: model adapters, LLM gateway hooks, vector DBs, tools & external APIs, and the platform’s LLMops flows.

• Build and improve tracing, benchmarking and observability for LLMs and agents - token/cost accounting, latency p95, throughput, and correctness checks.

• Drive design for safety/guardrails: moderation hooks, human-in-the-loop checkpoints, replayable audit trails and policy enforcement.

• Mentor junior engineers, run design reviews, and improve engineering practices (testing, CI/CD, chaos testing for agents).

• Work directly with strategic customers to prototype complex agentic solutions and translate them into product features.

Must-have

• 5–10 years of software engineering with substantial experience building distributed systems, infra, or ML platforms.

• Deep practical experience integrating and deploying LLMs in production (RAG, retrieval, embeddings pipelines).

• Hands-on experience with agent orchestration frameworks (LangGraph / LangChain or custom agent runtimes) and stateful workflow design.

• Strong systems knowledge: Kubernetes, container orchestration, service meshes, and performance tuning.

• Proven track record building observability, cost controls, and policy enforcement for production services.

Preferred / differentiators

• Experience building or contributing to open-source LLM orchestration tools (LangGraph, LangChain, or similar).

• Familiarity with enterprise constraints: on-prem/cloud hybrid deployments, data residency, compliance requirements.

• Background in security, privacy, or model governance for LLMs.

• Demonstrated leadership in cross-functional projects and direct customer engagement.

Traits we are looking for: ownership, ability to execute, hustle and thinking out of the box, data-driven decision making, be comfortable with more unknowns than knowns.

Perks of Working at TrueFoundry

Join a fast-growing Series A, Bay Area-based startup building cutting-edge AI infrastructure.
Comprehensive health insurance for you and your family, including medical, dental, and vision coverage.
401(k) retirement plan.
Flexible hybrid work, 2 days a week in the office (Tuesday & Wednesday), with flexibility around the schedule.
Lunch and snacks are on us whenever you come into the office.
Work from another TrueFoundry office for up to 1 month each year, whether that's our London or India office, if you'd like a change of scenery.

Apply About employer

More about this job

Skills

ML
LangChain
Cost accounting
CI/CD
Workflow
Kubernetes
Regulatory compliance
Leadership
Decision making

Report this job