Find work in the United States.

A simpler way to find your next job.

Jobs in the United States

25+ jobs

Save this search

Added 1 min ago
TT
Senior Software Engineer
True Talent Solutions Partners
San Francisco, CA
- $230,000 to $280,000 USD / year
Join a small engineering team at an AI data infrastructure company building a platform for automated data production pipelines. You will own complex product and architecture work end to…
Added 13 mins ago
O
Lead Principal Software Engineer, Core Infrastructure
Oracle
Nashville, TN
- $146,300 to $306,400 USD / year
…Oracle's Cloud Infrastructure. You should be both a rock-solid lead developer, curious problem solver, a distributed systems generalist and/or skilled Linux engineer with Systems triage…
Added 13 mins ago
O
Lead Principal Platform Software Engineer
Oracle
Seattle, WA · Santa Clara, CA
- $113,600 to $306,400 USD / year
We are seeking a Lead Principal Platform Software Engineer to lead the architecture and delivery of critical platform systems and initiatives across teams. This is a hands-on technical…
Added 26 mins ago
Senior Software Engineer (Institutional, Financing)
Coinbase
Trenton, NJ
As a Senior Software Engineer on the Financing Engineering team within the Institutional group, you'll design, build, and scale the systems that safely manage billions of dollars in loans…
Added 31 mins ago
WI
(USA) Principal, Software Engineer
Walmart Inc.
United States
- $143,000 to $286,000 USD / year
The (USA) Principal, Software Engineer leads the design and delivery of scalable platform capabilities, driving innovation in cloud-native architectures and AI/ML integration. This role…
Added 1 hour ago
Strategic Solutions Engineer
Procore Technologies, Inc.
TX
We are looking for a Strategic Solutions Engineer to serve as a technical subject matter expert and trusted customer advocate throughout the sales cycle in our strategic customer segment.…
Added 1 hour ago
Lead Senior Cloud Platform Engineer
Stanley Martin Homes, LLC
Reston, VA
A day in the life of a Lead Senior Cloud Platform Engineer begins with partnering with application, data, AI, and infrastructure teams to ensure cloud platforms are enabling the secure,…
Added 1 hour ago
G
Staff Software Engineer, Applied AI
GridUnity
Remote, United States
- $250,000 to $265,000 USD / year
GridUnity is seeking a Staff Software Engineer to join our Platform team, reporting to the Director of Engineering, Applied AI. You will turn our customer workflow vision into reliable…
Added 1 hour ago
UG
Principal Data Engineering - Remote
UnitedHealth Group
Minnetonka, MN
- $112,700 to $193,200 USD / year
The Principal Data Engineer is a senior individual contributor responsible for designing, building, and scaling enterprise data platforms and data pipelines. This role combines cloud-based…
Added 1 hour ago
UG
Senior Software Engineer
UnitedHealth Group
Eden Prairie, MN
- $91,700 to $163,700 USD / year
As a Senior Software Engineer on the OCPN Cloud Platform Team, you will drive the architecture, technical design, development, delivery, and operationalization of cloud-native, resilient,…
Added 1 hour ago
CB
Senior Data Engineer
Commercial Bank of California
United States
- $135,000 to $170,000 USD / year
The Senior Data Engineer is responsible for designing, building, and operating scalable, secure data platforms that power enterprise analytics, reporting, and advanced data use cases,…
Added 1 hour ago
A
Strategic Business Development Lead, Private Equity
Aleph
Remote, United States
- $159,800 to $225,000 USD / year
We’re looking for a Strategic Business Development Lead, Private Equity to help private equity firms and their portfolio companies evaluate Aleph. Think of this as a sales engineer role…
Added 1 hour ago
Software Developer
Koniag Government Services
Huntsville, AL
KGS is currently seeking an accomplished Senior Full Stack Software Engineer/Developer to spearhead the engineering, enhancement, and architectural modernization of core Line of Business…
Added 1 hour ago
Harness/Traceable Integration Engineer (Senior)
Koniag Government Services
Washington, DC
The ideal candidate is a highly experienced and technically authoritative DevSecOps and API security engineer with deep, hands-on expertise across the Harness platform portfolio - including…
Added 1 hour ago
Perplexity/AI Integration Lead
Koniag Government Services
Washington, DC
Serve as the subject matter expert and technical lead for AI platform integration initiatives, with a focus on Perplexity AI and other enterprise-grade LLM and generative AI tools.
Added 1 hour ago
Sr. AI Technical Lead & Data Developer (REMOTE)
Koniag Government Services
Albuquerque, NM
Koniag Professional Services, LLC, a Koniag Government Services company, is seeking a Sr. AI Technical Lead & Data Developer to support KPS and our government customer. The position remote.…
Added 1 hour ago
NPE Governance Study Lead Engineer (Study 2) (REMOTE)
Koniag Government Services
Washington, DC
Koniag IT Systems, LLC a Koniag Government Services company, is seeking a NPE Governance Study Lead Engineer (Study 2) with a Secret security clearance to support KITS and our government…
Added 1 hour ago
Principal Engineer - Core Platform Services
Target Enterprise Inc
NCD-0375 Brooklyn Park, MN
As a Principal Engineer, you are the senior-most individual-contributor technical leader on the Core Platform Services team. You partner with the Senior Engineering Manager and the team’s…
Added 1 hour ago
Sr Engineering Manager - Core Platform Services
Target Enterprise Inc
Minneapolis, MN
As a Senior Engineering Manager, you will lead the Core Platform Services team. You will partner closely with a Principal Engineer, the team’s Product Manager, and the leaders of peer…
Added 1 hour ago
Lead java engineer with aws
TransUnion LLC
Crum Lynne, PA
Added 1 hour ago
Principal Software Engineer
The MITRE Corporation
McLean, VA · Bedford, MA
…public interest. With objectivity and integrity at our core, we lead federally funded research and development centers for U.S. government agencies – collaborating with industry and…
Added 1 hour ago
A
Automation Engi
Actalent
Reading, PA
- $90,000 to $100,000 USD / year
Lead and coordinate teams to define user and functional requirements of the MES system and incorporate them into the MES system.
Added 1 hour ago
Full-Stack/UI Senior Software Engineer - 26375
Enverus
Pittsburgh, PA
We are the most trusted energy-dedicated SaaS company, with a platform built to maximize value from generative AI, and our innovative solutions are reshaping the way energy is consumed and…
Added 2 hours ago
TT
Product Engineer (Mid-Senior)
True Talent Solutions Partners
New York, NY
- $140,000 to $200,000 USD / year
…real customer workflows, while also owning broader product and platform work where it has the greatest…
Added 2 hours ago
Sr. CyberArk Implementation Engineer or Architect
Capgemini
Atlanta, GA, GA
We are seeking an experienced Senior CyberArk PAM Implementation Engineer to assess, optimize, and enhance our existing CyberArk Privileged Access Management environment. The ideal…

Select Minds LLC

Senior AI Tools Testing Engineer

Dallas, TX, United States

$120,000 to $140,000 USD / year

Apply About employer

Applications open in a new tab.

Job details

Pay: $120,000 to $140,000 USD / year
Posting language: English
Posted: Nov 5, 2026

Original job description

Job Title: Senior / Lead AI Tools & Testing Engineer (Agentic AI & LLM Evaluation) Location: Dallas, TX (Hybrid) Duration: Long‑Term Experience:10 + Years

We seek a senior/lead AI engineer with hands‑on experience building, orchestrating, monitoring, and evaluating production‑grade Generative AI and Agentic AI applications. The ideal candidate is expert in Amazon Bedrock, AgentCore, LangGraph, LangChain, RAG, LLM observability, and evaluation frameworks. This role focuses on creating reliable, scalable, and production‑ready AI applications, emphasizing agent orchestration, AI testing, LLM evaluation, and continuous quality monitoring.

The candidate will collaborate with engineering, QA, and platform teams to establish best practices for building, testing, monitoring, and improving AI‑powered applications.

Key Responsibilities

Agentic AI & GenAI Engineering

Design, develop, and deploy production‑grade agentic and generative AI applications.
Build and orchestrate AI agents using LangGraph, LangChain, Amazon Bedrock, and AgentCore.
Implement multi‑agent workflows, tool calling, state management, memory, and orchestration.
Create scalable LLM‑powered applications and reusable AI components.

RAG & LLM Application Development

Design and implement Retrieval‑Augmented Generation (RAG) solutions.
Develop pipelines for document ingestion, chunking, embedding, retrieval, reranking, and context management.
Optimize retrieval accuracy, response quality, latency, and cost.
Apply grounding, hallucination reduction, and context‑relevance techniques.

LLM Evaluation & AI Testing

Build evaluation frameworks for LLM‑based applications and agentic workflows.
Automate evaluation with DeepEval and other LLM evaluation tools.
Use LLM‑as‑a‑Judge methods to assess response quality, relevance, accuracy, faithfulness, and reasoning.
Create evaluation datasets, test scenarios, benchmarks, and regression‑testing pipelines.
Evaluate agent behavior, tool execution, workflow accuracy, and response consistency.
Identify failure patterns and continuously improve AI application quality.

AI Observability & Monitoring

Implement tracing, monitoring, and observability with OpenTelemetry, LangSmith, and Langfuse.
Track LLM requests, agent execution flows, tool calls, latency, token consumption, errors, and performance.
Establish debugging mechanisms for complex multi‑step agentic workflows.
Develop monitoring practices to detect performance degradation and operational issues.

AWS AI Platform & Production Engineering

Deploy AI applications using Amazon Bedrock and Bedrock AgentCore.
Integrate foundation models, agent frameworks, APIs, and enterprise data sources.
Apply production‑readiness practices covering scalability, reliability, security, and performance.
Support CI/CD integration and automated evaluation of AI applications.
Optimize model selection, inference costs, latency, and resource utilization.

Technical Leadership & Collaboration

Provide technical guidance to engineering and QA teams.
Establish reusable frameworks, implementation patterns, and best practices.
Collaborate with architecture, DevOps, QA, and engineering teams.
Participate in design discussions, code reviews, proof‑of‑concept development, and implementation.
Mentor engineers and help teams adopt effective AI development and testing practices.

Required Technical Skills

Agentic AI & Orchestration

LangGraph
LangChain
Amazon Bedrock
Amazon Bedrock AgentCore
AI agents, multi‑agent orchestration, tool calling, workflow management

Generative AI & RAG

Large Language Models (LLMs)
Retrieval‑Augmented Generation (RAG)
Embeddings and vector search
Prompt engineering and context management
LLM application development

LLM Evaluation & Testing

DeepEval
LLM‑as‑a‑Judge techniques
Evaluation metrics and benchmarking
Automated evaluation and regression testing
Assessment of hallucination, faithfulness, relevance, and response quality

AI Observability & Monitoring

OpenTelemetry
LangSmith
Langfuse
LLM tracing, debugging, and performance monitoring

Production Engineering

Deploying and operating production‑grade GenAI applications
API integration and backend engineering
CI/CD and automated testing integration
Scalability, reliability, security, and performance optimization

Preferred Qualifications

8+ years of software engineering or IT experience.
Hands‑on experience developing and deploying GenAI or Agentic AI applications.
Experience with enterprise‑scale AI implementations.
Familiarity with AWS cloud services and enterprise AI security practices.
Experience establishing AI quality standards and evaluation methodologies.
Strong problem‑solving, communication, and technical collaboration skills.

What Success Looks Like

Development and deployment of production‑grade agentic AI applications.
Reliable orchestration and execution of AI agents and workflows.
Effective LLM evaluation and automated AI testing frameworks.
Improved AI application accuracy, reliability, observability, and performance.
Reusable engineering practices for building and maintaining enterprise GenAI solutions.

Important: This is a hands‑on engineering position. Candidates with practical experience building and evaluating AI applications are preferred over those whose experience is limited to AI strategy, architecture documentation, conventional QA automation, or using AI coding assistants.

Flexible work from home options available.

Apply About employer

More about this job

Skills

LangChain
Workflow
Debugging
AWS
CI/CD
Test automation
Problem solving
Communication skills
Collaboration

Report this job