Find work in the United States.

A simpler way to find your next job.

Jobs in the United States

25+ jobs

Save this search

Added 1 min ago
Senior Vice President of Client Relationships
TAX MANAGEMENT ASSOCIATES INC
Nashville, TN · Indianapolis, IN · Charlotte, NC
Lead all aspects of the client-facing lifecycle, including post-sale transition, project communications, milestone visibility, issue resolution, completion, renewal readiness, and…
Added 1 min ago
SL
Office Manager, International Student Services
Salt Lake Community College (UT)
Salt Lake City, UT
- $60,147.47 to $65,377.69 USD / year
Serve as the departmental lead for administrative systems, including ISSM, appointment scheduling platforms, shared mailboxes, and departmental records management systems.
Added 6 mins ago
G
Key Account Manager- Pet Specialty
GPAC
Douglasville, GA
- $80,000 to $100,000 USD / year
Analyze historical and current purchase order data to identify growth opportunities
Added 6 mins ago
G
Key Account Manager- Pet Specialty
GPAC
Mableton, GA
- $80,000 to $100,000 USD / year
Analyze historical and current purchase order data to identify growth opportunities
Added 7 mins ago
G
Key Account Manager- Pet Specialty
GPAC
Atlanta, GA
- $80,000 to $100,000 USD / year
Analyze historical and current purchase order data to identify growth opportunities
Added 7 mins ago
G
Key Account Manager- Pet Specialty
GPAC
Powder Springs, GA
- $80,000 to $100,000 USD / year
Analyze historical and current purchase order data to identify growth opportunities
Added 7 mins ago
G
Key Account Manager- Pet Specialty
GPAC
Canton, GA
- $80,000 to $100,000 USD / year
Analyze historical and current purchase order data to identify growth opportunities
Added 7 mins ago
G
Key Account Manager- Pet Specialty
GPAC
Dallas, GA
- $80,000 to $100,000 USD / year
Analyze historical and current purchase order data to identify growth opportunities
Added 7 mins ago
G
Key Account Manager- Pet Specialty
GPAC
Snellville, GA
- $80,000 to $100,000 USD / year
Analyze historical and current purchase order data to identify growth opportunities
Added 7 mins ago
G
Key Account Manager- Pet Specialty
GPAC
Clarkston, GA
- $80,000 to $100,000 USD / year
Analyze historical and current purchase order data to identify growth opportunities
Added 7 mins ago
G
Key Account Manager- Pet Specialty
GPAC
Austell, GA
- $80,000 to $100,000 USD / year
Analyze historical and current purchase order data to identify growth opportunities
Added 7 mins ago
G
Key Account Manager- Pet Specialty
GPAC
Avondale Estates, GA
- $80,000 to $100,000 USD / year
Analyze historical and current purchase order data to identify growth opportunities
Added 7 mins ago
G
Electrical Construction Project Manager
GPAC
Goodyear, AZ
- $110,000 to $140,000 USD / year
Lead multiple commercial electrical construction projects from kickoff through closeout
Added 8 mins ago
Director, Operational Resilience
NAC Voya Services Company
Atlanta, GA · SC-Work@Home, SC · CT-Work@Home, CT · NY-Work@Home, NY · MA-Work@Home, MA
The Director, Operational Resilience will lead the implementation and maturation of Voya's enterprise Operational Resilience Framework, ensuring the organization can continue delivering…
Added 11 mins ago
GI
Senior Data Strategy & Analytics Specialist
Genentech, Inc.
United States
- $107,400 to $199,400 USD / year
The Senior Data Strategy & Analytics Specialist plays a key role in designing and executing end-to-end data strategies and complex analytics that enable decision-making, asset strategy, and…
Added 12 mins ago
J
Sr. Content Strategist
Jobgether
Remote, United States
- $100,000 to $110,000 USD / year
The Senior Content Strategist will lead the content strategy and governance of a large-scale public data platform supporting a federal health agency.
Added 12 mins ago
Senior Site Reliability Engineer
Jobgether
Remote, United States
As a Senior Site Reliability Engineer, you will lead initiatives to strengthen infrastructure resilience, optimize application performance, and ensure the continuous delivery of reliable…
Added 12 mins ago
J
SENIOR NETWORK SECURITY ENGINEER
Jobgether
Remote, United States
- $120,000 to $140,000 USD / year
The Senior Network Security Engineer will lead the design, deployment, and maintenance of secure network access solutions across cloud and on-premises environments. This role focuses on…
Added 12 mins ago
Senior Director, Pharma Toxicology
Jobgether
Remote, United States
…development through registration and lifecycle management. You will lead and develop a high-performing toxicology organization while building strong relationships with internal…
Added 12 mins ago
Senior Director of Finance
Jobgether
Remote, United States
The Senior Director of Finance will lead the financial strategy, planning, accounting, and operations of a growing subscription-based technology business. This leadership role will work…
Added 12 mins ago
Consumer Marketing - Growth Lead
Jobgether
Remote, United States
As a Consumer Marketing - Growth Lead, you will drive the growth and success of a direct-to-consumer business in the rapidly evolving Physical AI industry. You will own the end-to-end…
Added 12 mins ago
J
Associate Director, AI & Applied Solutions
Jobgether
Remote, United States
- $140,000 to $190,000 USD / year
The Associate Director, AI & Applied Solutions will lead the practical application of artificial intelligence, data, and data science to solve complex business challenges and deliver…
Added 12 mins ago
J
Account Manager
Jobgether
Remote, United States
- $140,000 to $160,000 USD / year
In this role, you will lead strategic sales initiatives, develop customer relationships, and identify opportunities to expand the adoption of investigative intelligence solutions across…
Added 12 mins ago
Account Executive
Jobgether
Remote, United States
…relationships with organizations seeking advanced cybersecurity and AI data protection solutions. This role focuses on identifying new business opportunities, expanding key accounts, and…
Added 12 mins ago
GD
Joint Live Virtual Constructive-Gaming Technician Sr – SECRET clearance required
General Dynamics Information Technology
Schofield, HI
- $30.65 to $41.47 USD / hour
Must be able to lead a technical team to create a high fidelity, realistic training event in a Simulated and Constructive environment to meet customer requirements for distributed and…

Select Minds LLC

Senior AI Tools Testing Engineer

Dallas, TX, United States

$120,000 to $140,000 USD / year

Apply About employer

Applications open in a new tab.

Job details

Pay: $120,000 to $140,000 USD / year
Posting language: English
Posted: Nov 5, 2026

Original job description

Job Title: Senior / Lead AI Tools & Testing Engineer (Agentic AI & LLM Evaluation) Location: Dallas, TX (Hybrid) Duration: Long‑Term Experience:10 + Years

We seek a senior/lead AI engineer with hands‑on experience building, orchestrating, monitoring, and evaluating production‑grade Generative AI and Agentic AI applications. The ideal candidate is expert in Amazon Bedrock, AgentCore, LangGraph, LangChain, RAG, LLM observability, and evaluation frameworks. This role focuses on creating reliable, scalable, and production‑ready AI applications, emphasizing agent orchestration, AI testing, LLM evaluation, and continuous quality monitoring.

The candidate will collaborate with engineering, QA, and platform teams to establish best practices for building, testing, monitoring, and improving AI‑powered applications.

Key Responsibilities

Agentic AI & GenAI Engineering

Design, develop, and deploy production‑grade agentic and generative AI applications.
Build and orchestrate AI agents using LangGraph, LangChain, Amazon Bedrock, and AgentCore.
Implement multi‑agent workflows, tool calling, state management, memory, and orchestration.
Create scalable LLM‑powered applications and reusable AI components.

RAG & LLM Application Development

Design and implement Retrieval‑Augmented Generation (RAG) solutions.
Develop pipelines for document ingestion, chunking, embedding, retrieval, reranking, and context management.
Optimize retrieval accuracy, response quality, latency, and cost.
Apply grounding, hallucination reduction, and context‑relevance techniques.

LLM Evaluation & AI Testing

Build evaluation frameworks for LLM‑based applications and agentic workflows.
Automate evaluation with DeepEval and other LLM evaluation tools.
Use LLM‑as‑a‑Judge methods to assess response quality, relevance, accuracy, faithfulness, and reasoning.
Create evaluation datasets, test scenarios, benchmarks, and regression‑testing pipelines.
Evaluate agent behavior, tool execution, workflow accuracy, and response consistency.
Identify failure patterns and continuously improve AI application quality.

AI Observability & Monitoring

Implement tracing, monitoring, and observability with OpenTelemetry, LangSmith, and Langfuse.
Track LLM requests, agent execution flows, tool calls, latency, token consumption, errors, and performance.
Establish debugging mechanisms for complex multi‑step agentic workflows.
Develop monitoring practices to detect performance degradation and operational issues.

AWS AI Platform & Production Engineering

Deploy AI applications using Amazon Bedrock and Bedrock AgentCore.
Integrate foundation models, agent frameworks, APIs, and enterprise data sources.
Apply production‑readiness practices covering scalability, reliability, security, and performance.
Support CI/CD integration and automated evaluation of AI applications.
Optimize model selection, inference costs, latency, and resource utilization.

Technical Leadership & Collaboration

Provide technical guidance to engineering and QA teams.
Establish reusable frameworks, implementation patterns, and best practices.
Collaborate with architecture, DevOps, QA, and engineering teams.
Participate in design discussions, code reviews, proof‑of‑concept development, and implementation.
Mentor engineers and help teams adopt effective AI development and testing practices.

Required Technical Skills

Agentic AI & Orchestration

LangGraph
LangChain
Amazon Bedrock
Amazon Bedrock AgentCore
AI agents, multi‑agent orchestration, tool calling, workflow management

Generative AI & RAG

Large Language Models (LLMs)
Retrieval‑Augmented Generation (RAG)
Embeddings and vector search
Prompt engineering and context management
LLM application development

LLM Evaluation & Testing

DeepEval
LLM‑as‑a‑Judge techniques
Evaluation metrics and benchmarking
Automated evaluation and regression testing
Assessment of hallucination, faithfulness, relevance, and response quality

AI Observability & Monitoring

OpenTelemetry
LangSmith
Langfuse
LLM tracing, debugging, and performance monitoring

Production Engineering

Deploying and operating production‑grade GenAI applications
API integration and backend engineering
CI/CD and automated testing integration
Scalability, reliability, security, and performance optimization

Preferred Qualifications

8+ years of software engineering or IT experience.
Hands‑on experience developing and deploying GenAI or Agentic AI applications.
Experience with enterprise‑scale AI implementations.
Familiarity with AWS cloud services and enterprise AI security practices.
Experience establishing AI quality standards and evaluation methodologies.
Strong problem‑solving, communication, and technical collaboration skills.

What Success Looks Like

Development and deployment of production‑grade agentic AI applications.
Reliable orchestration and execution of AI agents and workflows.
Effective LLM evaluation and automated AI testing frameworks.
Improved AI application accuracy, reliability, observability, and performance.
Reusable engineering practices for building and maintaining enterprise GenAI solutions.

Important: This is a hands‑on engineering position. Candidates with practical experience building and evaluating AI applications are preferred over those whose experience is limited to AI strategy, architecture documentation, conventional QA automation, or using AI coding assistants.

Flexible work from home options available.

Apply About employer

More about this job

Skills

LangChain
Workflow
Debugging
AWS
CI/CD
Test automation
Problem solving
Communication skills
Collaboration

Report this job