Find work in the United States.

A simpler way to find your next job.

Jobs in the United States

25+ jobs

Save this search

Added just now
Data Operations Site Lead
Maxinsights
Santa Clara, CA
…algorithm-ready dataset delivery - for leading robotics foundation model companies and embodied AI labs…
Added 1 min ago
Senior Product Manager - AI & Healthcare Innovation
Evolent
Jefferson City, MO
+ Partner with data science and engineering leaders to define scalable AI architectures, model evaluation frameworks, and production readiness criteria.
Added 1 min ago
U
Software Engineer II - Engineer
Uber
San Francisco, CA · Sunnyvale, CA
- $171,000 to $190,000 USD / year
Own and develop on all parts of the ML lifecycle - both offline (training, model architecture, etc.) and online (serving efficiency, GPU provisioning + utilization, etc.).
Added 2 mins ago
Automation and AI Enablement Engineer
FiberFirst
Southlake, TX
…with sound judgment about when to automate, when to apply a model, and when to leave a working system…
Added 3 mins ago
Senior Ads Sales Account Manager
N3, LLC Company
United States
Understand each client’s business model, audience, commercial objectives, and marketing priorities, translating these into tailored advertising strategies.
Added 3 mins ago
Forward Deployed Engineer - Databricks - Forensic Discovery & Financial Crime
Deloitte
Saint Louis, MO
…generative artificial intelligence (GenAI), or large language model (LLM)-powered solutions in client or production…
Added 3 mins ago
Forward Deployed Engineer - Databricks - Forensic Discovery & Financial Crime
Deloitte
Kansas City, MO
…generative artificial intelligence (GenAI), or large language model (LLM)-powered solutions in client or production…
Added 3 mins ago
SAP Basis/Technology Senior Manager - Resources
Accenture LLP Company
NY
You have experience with SAP RISE commercial model - understanding shared responsibility boundaries, SAP-managed vs. customer-managed scope, and implications for AMS contract structuring.
Added 3 mins ago
Salesforce Data Cloud & AI Tech Architect (Tech Architecture/Solution Architecture)
Accenture
Saint Louis, MO
+ Solution design ownership - lead end-to-end solution design across assigned workstreams, including data model, integration patterns, security model, and platform configuration; keep SDDs,…
Added 3 mins ago
Sales Transformation Consultant | Financial Services
Accenture LLP Company
United States
Design and implement business changes that drive industry-specific, function and digital operating model transformation, focusing on tasks relating to people, process and technology.
Added 3 mins ago
Knowledge Engineer / Semantic Expert for AI
Accenture LLP Company
NY
You lead a team and provide guidance to explore and implement new methodologies, model building techniques, and cutting-edge algorithms, and applying these techniques with the right…
Added 4 mins ago
Salesforce Data Cloud & AI Architect Senior Manager
Accenture
Saint Louis, MO
+ Data Cloud architecture - own end-to-end design of Data Cloud solutions including data model, ingestion strategy, identity resolution, unified profiles, segmentation, and real-time…
Added 4 mins ago
Talent & Workforce Consulting Sr. Manager, Chemicals & Natural Resources
Accenture
Saint Louis, MO
+ Apply human-centered approaches to integrate AI, automation, and emerging technologies into workforce, talent, and operating model solutions.
Added 5 mins ago
Director, Enterprise Content Strategy & Newsrooms
The Cigna Group
Saint Louis, MO
+ Experience overseeing multiple owned-content properties or a federated content model.
Added 5 mins ago
Senior Technical Marketing Engineer, Distributed Networking, Content Delivery & AI
Akamai Technologies, Inc.
Jefferson City, MO
…telco peering, route optimization, post quantum cryptography, developer experience, performance, and the networking patterns emerging and around AI inference and…
Added 5 mins ago
A
Senior Global Recognition Technology Manager
Auror
Denver, CO
- $155,000 to $180,000 USD / year
This is a Denver, Colorado-based role; we value co-location and have a hybrid flexible working model. Our recognition technology product modules are leveraged by customers in New Zealand,…
Added 5 mins ago
Talent & Workforce Consulting Senior Manager, Energy
Accenture
Saint Louis, MO
- Apply human-centered approaches to integrate AI, automation, and emerging technologies into workforce, talent, and operating model solutions.
Added 5 mins ago
Talent & Workforce Consulting Manager, Energy
Accenture
Saint Louis, MO
+ Apply human-centered approaches to integrate AI, automation, and emerging technologies into workforce, talent, and operating model solutions.
Added 6 mins ago
C
Entry-Level Engineer - Python, Data Analytics & Generative AI Life Sciences Commercial
Cognizant
Jefferson City, MO
- $65,000 USD / year
+ Understanding of MLOps or model deployment concepts
Added 6 mins ago
Tax Partner
Cherry Bekaert Advisory LLC
Houston, TX
You’ll use your client service team leadership skills, acting as a role model, to provide coaching, mentoring, and training in the development of managers and staff.
Added 8 mins ago
Talent & Workforce Consulting Manager - Utilities
Accenture
Saint Louis, MO
…including change management, leadership development, and operating model…
Added 9 mins ago
AT
Senior II Site Reliability Engineer - Network
Akamai Technologies, Inc.
Jefferson City, MO
- $146,400 to $263,600 USD / year
+ Have an understanding of networking such as TCP/IP, OSI model, and Cisco or Juniper routers and switches
Added 9 mins ago
Resident Piping Engineer - Micron
Bechtel Corporation
Clay, NY
Experience with multi discipline interface management and global 3D model oversight.
Added 11 mins ago
SAP - Oil & Gas - Retail Fuel - Manager - Location OPEN
EY
Kansas City, MO
+ Join us in our team-led and leader-enabled hybrid model. Our expectation is for most people in external, client serving roles to work together in person 40-60% of the time over the course…
Added 11 mins ago
SAP - Oil & Gas - Retail Fuel - Manager - Location OPEN
EY
Saint Louis, MO
+ Join us in our team-led and leader-enabled hybrid model. Our expectation is for most people in external, client serving roles to work together in person 40-60% of the time over the course…

Select Minds LLC

Senior AI Tools Testing Engineer

Dallas, TX, United States

$120,000 to $140,000 USD / year

Apply About employer

Applications open in a new tab.

Job details

Pay: $120,000 to $140,000 USD / year
Posting language: English
Posted: Nov 5, 2026

Original job description

Job Title: Senior / Lead AI Tools & Testing Engineer (Agentic AI & LLM Evaluation) Location: Dallas, TX (Hybrid) Duration: Long‑Term Experience:10 + Years

We seek a senior/lead AI engineer with hands‑on experience building, orchestrating, monitoring, and evaluating production‑grade Generative AI and Agentic AI applications. The ideal candidate is expert in Amazon Bedrock, AgentCore, LangGraph, LangChain, RAG, LLM observability, and evaluation frameworks. This role focuses on creating reliable, scalable, and production‑ready AI applications, emphasizing agent orchestration, AI testing, LLM evaluation, and continuous quality monitoring.

The candidate will collaborate with engineering, QA, and platform teams to establish best practices for building, testing, monitoring, and improving AI‑powered applications.

Key Responsibilities

Agentic AI & GenAI Engineering

Design, develop, and deploy production‑grade agentic and generative AI applications.
Build and orchestrate AI agents using LangGraph, LangChain, Amazon Bedrock, and AgentCore.
Implement multi‑agent workflows, tool calling, state management, memory, and orchestration.
Create scalable LLM‑powered applications and reusable AI components.

RAG & LLM Application Development

Design and implement Retrieval‑Augmented Generation (RAG) solutions.
Develop pipelines for document ingestion, chunking, embedding, retrieval, reranking, and context management.
Optimize retrieval accuracy, response quality, latency, and cost.
Apply grounding, hallucination reduction, and context‑relevance techniques.

LLM Evaluation & AI Testing

Build evaluation frameworks for LLM‑based applications and agentic workflows.
Automate evaluation with DeepEval and other LLM evaluation tools.
Use LLM‑as‑a‑Judge methods to assess response quality, relevance, accuracy, faithfulness, and reasoning.
Create evaluation datasets, test scenarios, benchmarks, and regression‑testing pipelines.
Evaluate agent behavior, tool execution, workflow accuracy, and response consistency.
Identify failure patterns and continuously improve AI application quality.

AI Observability & Monitoring

Implement tracing, monitoring, and observability with OpenTelemetry, LangSmith, and Langfuse.
Track LLM requests, agent execution flows, tool calls, latency, token consumption, errors, and performance.
Establish debugging mechanisms for complex multi‑step agentic workflows.
Develop monitoring practices to detect performance degradation and operational issues.

AWS AI Platform & Production Engineering

Deploy AI applications using Amazon Bedrock and Bedrock AgentCore.
Integrate foundation models, agent frameworks, APIs, and enterprise data sources.
Apply production‑readiness practices covering scalability, reliability, security, and performance.
Support CI/CD integration and automated evaluation of AI applications.
Optimize model selection, inference costs, latency, and resource utilization.

Technical Leadership & Collaboration

Provide technical guidance to engineering and QA teams.
Establish reusable frameworks, implementation patterns, and best practices.
Collaborate with architecture, DevOps, QA, and engineering teams.
Participate in design discussions, code reviews, proof‑of‑concept development, and implementation.
Mentor engineers and help teams adopt effective AI development and testing practices.

Required Technical Skills

Agentic AI & Orchestration

LangGraph
LangChain
Amazon Bedrock
Amazon Bedrock AgentCore
AI agents, multi‑agent orchestration, tool calling, workflow management

Generative AI & RAG

Large Language Models (LLMs)
Retrieval‑Augmented Generation (RAG)
Embeddings and vector search
Prompt engineering and context management
LLM application development

LLM Evaluation & Testing

DeepEval
LLM‑as‑a‑Judge techniques
Evaluation metrics and benchmarking
Automated evaluation and regression testing
Assessment of hallucination, faithfulness, relevance, and response quality

AI Observability & Monitoring

OpenTelemetry
LangSmith
Langfuse
LLM tracing, debugging, and performance monitoring

Production Engineering

Deploying and operating production‑grade GenAI applications
API integration and backend engineering
CI/CD and automated testing integration
Scalability, reliability, security, and performance optimization

Preferred Qualifications

8+ years of software engineering or IT experience.
Hands‑on experience developing and deploying GenAI or Agentic AI applications.
Experience with enterprise‑scale AI implementations.
Familiarity with AWS cloud services and enterprise AI security practices.
Experience establishing AI quality standards and evaluation methodologies.
Strong problem‑solving, communication, and technical collaboration skills.

What Success Looks Like

Development and deployment of production‑grade agentic AI applications.
Reliable orchestration and execution of AI agents and workflows.
Effective LLM evaluation and automated AI testing frameworks.
Improved AI application accuracy, reliability, observability, and performance.
Reusable engineering practices for building and maintaining enterprise GenAI solutions.

Important: This is a hands‑on engineering position. Candidates with practical experience building and evaluating AI applications are preferred over those whose experience is limited to AI strategy, architecture documentation, conventional QA automation, or using AI coding assistants.

Flexible work from home options available.

Apply About employer

More about this job

Skills

LangChain
Workflow
Debugging
AWS
CI/CD
Test automation
Problem solving
Communication skills
Collaboration

Report this job