Find work in the United States.

A simpler way to find your next job.

Jobs in the United States

25+ jobs

Save this search

Added 2 mins ago
Senior AI Engineer
Suffolk Construction Company, Inc
Boston, MA
Own full lifecycle delivery - design, development, testing, deployment, monitoring, and maintenance.
Added 4 mins ago
Mechanical Design & Analysis Engineer
Qnity
Chesterfield, MO
The Mechanical Design & Analysis Engineer is responsible for, but not limited to, design and manufacturing-related functions required by key strategic business unit account customers. This…
Added 8 mins ago
AI Engineer 4 (AI Safety and Multimodal)
Capital One
San Jose, CA · McLean, VA · Cambridge, MA · NY
Experience leading development AI systems with tradeoff decisions around cost, latency, throughput and accuracy
Added 8 mins ago
Senior Software Engineer - Frontend - Coinbase Card team
Coinbase
Jefferson City, MO
Frontend Software Engineer - Coinbase Card Team
Added 9 mins ago
Staff AI Engineer
Nova Biomedical
Waltham, MA
The Staff AI Engineer will serve as a senior hands-on technical leader within Nova's Data & AI function, designing, building, and operationalizing AI-enabled applications, agents, reusable…
Added 9 mins ago
Digital Workplace Experience Engineer
Nova Biomedical
Waltham, MA
…to support employees, manufacturing operations, research and development, commercial teams, and corporate functions across a global…
Added 9 mins ago
CC
Sr Quality Engineer
Carrier Corporation
Collierville, TN
- $79,000 to $158,000 USD / year
…operations standards as our Collierville facility’s senior quality engineer. This position offers high visibility and an excellent pathway to future…
Added 9 mins ago
CC
Staff Site Reliability Engineer
Carrier Corporation
Remote, United States
- $96,000 to $192,000 USD / year
Partner with development teams to improve application reliability throughout the software development lifecycle.
Added 9 mins ago
STAFF SOFTWARE ENGINEER
The Home Depot
Remote, United States
…infrastructure-as-code tools including Terraform and cdk8s (Cloud Development Kit for Kubernetes written in TypeScript) to provision and manage cloud resources and Kubernetes workloads…
Added 10 mins ago
Mechanical engineer
ACS Acquisitions, Inc
CA
To support our extraordinary teams who build great products and contribute to our growth, we're looking to add a Mechanical Engineer located in Milpitas, California.
Added 11 mins ago
AI & Distributed Systems Fullstack Engineer - Senior - Consulting - Location OPEN
EY
Saint Louis, MO
+ Proficiency in application development tools and quality assurance practices.
Added 11 mins ago
AI & Distributed Systems Fullstack Engineer - Senior - Consulting - Location OPEN
EY
Kansas City, MO
+ Proficiency in application development tools and quality assurance practices.
Added 11 mins ago
AI & Distributed Systems Fullstack Engineer - Senior - Consulting - Location OPEN
EY
Jefferson City, MO
+ Proficiency in application development tools and quality assurance practices.
Added 13 mins ago
A
Geotechnical Engineer
Actalent
Kirkland, WA
- $35 to $45.6 USD / hour
[Job Description]An Engineering company located in Kirkland, WA is currently hiring for a Geotechnical Engineer to support and, depending on experience, manage geotechnical projects from…
Added 14 mins ago
WW
Observability & SRE Engineer
World Wide Technology
Maryland Heights, MO
- $90,000 to $135,000 USD / year
World Wide Technology (WWT) is seeking an Engineer to join the Observability & Site Reliability Engineering (SRE) team. In this role, the Engineer will design, build, and operate the…
Added 14 mins ago
Senior AI Engineer, Big Data & AI Solutions
Mastercard
O'Fallon, MO
As a Senior AI Engineer, you will work at the intersection of big data engineering and AI/ML, helping build and support the data, features, models, and production processes behind fraud…
Added 14 mins ago
Software Test and Automation Engineer
Edwards Lifesciences LLC
CA
…suite: digital ecosystem applications (mobile and web), the application platform (microservices in .NET Core, Java, Python, GoLang, Rust), and the data platform (Databricks components and…
Added 14 mins ago
Senior Engineer - Global Rates eTrading Tech
Bank of America, National Association
United States
Bank of America is committed to an in-office culture that supports collaboration, engagement, and career development. Our approach includes clear in-office expectations, while providing an…
Added 16 mins ago
Senior Security Engineer - Secure SDLC
Highmark Health
Jefferson City, MO
Highmark Health is seeking a Senior Security Engineer to join our Enterprise Application Security team and play a pivotal role in shaping how security is built into our software - not…
Added 19 mins ago
Senior System Protection Engineer 2 - Grid
Sargent & Lundy
Clayton, MO
Lead the development and application of protection philosophies and settings for:
Added 19 mins ago
Lead System Protection Engineer 2 - Grid
Sargent & Lundy
Clayton, MO
Lead the development and application of protection philosophies and settings for:
Added 23 mins ago
Instructional Designer I or II
Texas A&M University
College Station, TX
…lives through basic and applied engineering research, workforce development and technology transition. Our collaborations with industry, academia and government provide cutting-edge…
Added 23 mins ago
Director, Packaging & Monetization Strategy - Employee Experience Group
ServiceNow
Mountain View, CA
It all started when engineer Fred Luddy wrote code that automated a tedious task for his coworker, Phyllis. She cried tears of joy. That moment inspired Fred to build a company that could…
Added 24 mins ago
Citrix Software Virtualization Engineer - Top Secret
General Dynamics Information Technology
Saint Louis, MO
At GDIT, people are our differentiator. As a Citrix Software Virtualization Engineer you will help ensure today is safe and tomorrow is smarter. Our work depends on a TS/SCI Cleared Citrix…
Added 25 mins ago
T
PCB Librarian
TEKsystems
Round Rock, TX
- $75 to $77.46 USD / hour
We are seeking a PCB Librarian/eCAD Library Engineer to support the expansion of its internal hardware development organization. This individual will build, validate, and maintain…

Select Minds LLC

Senior AI Tools Testing Engineer

Dallas, TX, United States

$120,000 to $140,000 USD / year

Apply About employer

Applications open in a new tab.

Job details

Pay: $120,000 to $140,000 USD / year
Posting language: English
Posted: Nov 5, 2026

Original job description

Job Title: Senior / Lead AI Tools & Testing Engineer (Agentic AI & LLM Evaluation) Location: Dallas, TX (Hybrid) Duration: Long‑Term Experience:10 + Years

We seek a senior/lead AI engineer with hands‑on experience building, orchestrating, monitoring, and evaluating production‑grade Generative AI and Agentic AI applications. The ideal candidate is expert in Amazon Bedrock, AgentCore, LangGraph, LangChain, RAG, LLM observability, and evaluation frameworks. This role focuses on creating reliable, scalable, and production‑ready AI applications, emphasizing agent orchestration, AI testing, LLM evaluation, and continuous quality monitoring.

The candidate will collaborate with engineering, QA, and platform teams to establish best practices for building, testing, monitoring, and improving AI‑powered applications.

Key Responsibilities

Agentic AI & GenAI Engineering

Design, develop, and deploy production‑grade agentic and generative AI applications.
Build and orchestrate AI agents using LangGraph, LangChain, Amazon Bedrock, and AgentCore.
Implement multi‑agent workflows, tool calling, state management, memory, and orchestration.
Create scalable LLM‑powered applications and reusable AI components.

RAG & LLM Application Development

Design and implement Retrieval‑Augmented Generation (RAG) solutions.
Develop pipelines for document ingestion, chunking, embedding, retrieval, reranking, and context management.
Optimize retrieval accuracy, response quality, latency, and cost.
Apply grounding, hallucination reduction, and context‑relevance techniques.

LLM Evaluation & AI Testing

Build evaluation frameworks for LLM‑based applications and agentic workflows.
Automate evaluation with DeepEval and other LLM evaluation tools.
Use LLM‑as‑a‑Judge methods to assess response quality, relevance, accuracy, faithfulness, and reasoning.
Create evaluation datasets, test scenarios, benchmarks, and regression‑testing pipelines.
Evaluate agent behavior, tool execution, workflow accuracy, and response consistency.
Identify failure patterns and continuously improve AI application quality.

AI Observability & Monitoring

Implement tracing, monitoring, and observability with OpenTelemetry, LangSmith, and Langfuse.
Track LLM requests, agent execution flows, tool calls, latency, token consumption, errors, and performance.
Establish debugging mechanisms for complex multi‑step agentic workflows.
Develop monitoring practices to detect performance degradation and operational issues.

AWS AI Platform & Production Engineering

Deploy AI applications using Amazon Bedrock and Bedrock AgentCore.
Integrate foundation models, agent frameworks, APIs, and enterprise data sources.
Apply production‑readiness practices covering scalability, reliability, security, and performance.
Support CI/CD integration and automated evaluation of AI applications.
Optimize model selection, inference costs, latency, and resource utilization.

Technical Leadership & Collaboration

Provide technical guidance to engineering and QA teams.
Establish reusable frameworks, implementation patterns, and best practices.
Collaborate with architecture, DevOps, QA, and engineering teams.
Participate in design discussions, code reviews, proof‑of‑concept development, and implementation.
Mentor engineers and help teams adopt effective AI development and testing practices.

Required Technical Skills

Agentic AI & Orchestration

LangGraph
LangChain
Amazon Bedrock
Amazon Bedrock AgentCore
AI agents, multi‑agent orchestration, tool calling, workflow management

Generative AI & RAG

Large Language Models (LLMs)
Retrieval‑Augmented Generation (RAG)
Embeddings and vector search
Prompt engineering and context management
LLM application development

LLM Evaluation & Testing

DeepEval
LLM‑as‑a‑Judge techniques
Evaluation metrics and benchmarking
Automated evaluation and regression testing
Assessment of hallucination, faithfulness, relevance, and response quality

AI Observability & Monitoring

OpenTelemetry
LangSmith
Langfuse
LLM tracing, debugging, and performance monitoring

Production Engineering

Deploying and operating production‑grade GenAI applications
API integration and backend engineering
CI/CD and automated testing integration
Scalability, reliability, security, and performance optimization

Preferred Qualifications

8+ years of software engineering or IT experience.
Hands‑on experience developing and deploying GenAI or Agentic AI applications.
Experience with enterprise‑scale AI implementations.
Familiarity with AWS cloud services and enterprise AI security practices.
Experience establishing AI quality standards and evaluation methodologies.
Strong problem‑solving, communication, and technical collaboration skills.

What Success Looks Like

Development and deployment of production‑grade agentic AI applications.
Reliable orchestration and execution of AI agents and workflows.
Effective LLM evaluation and automated AI testing frameworks.
Improved AI application accuracy, reliability, observability, and performance.
Reusable engineering practices for building and maintaining enterprise GenAI solutions.

Important: This is a hands‑on engineering position. Candidates with practical experience building and evaluating AI applications are preferred over those whose experience is limited to AI strategy, architecture documentation, conventional QA automation, or using AI coding assistants.

Flexible work from home options available.

Apply About employer

More about this job

Skills

LangChain
Workflow
Debugging
AWS
CI/CD
Test automation
Problem solving
Communication skills
Collaboration

Report this job