Find work in the United States.

A simpler way to find your next job.

Jobs in the United States

25+ jobs

Save this search

Added 3 mins ago
Senior Telecom Engineer
Maximus
Des Moines, IA
Maximus is seeking an experienced Senior Telecom Engineer to lead the design, implementation, and optimization of Genesys Cloud solutions. This role is responsible for executing large-scale…
Added 3 mins ago
Senior Design Manager (Data Center)
Jacobs
Davenport, IA
…technology cooling water interfaces, leak detection, and thermal…
Added 3 mins ago
AA
Lead Data Analytics Engineer
Aegon Asset Management
Cedar Rapids, IA
- $120,000 to $130,000 USD / year
Experience with Microsoft Fabric, including pipelines, OneLake, and data synchronization optimization
Added 3 mins ago
CH
Data Analyst
Cayuse Holdings
Des Moines, IA
- $100,000 to $110,000 USD / year
The Data Engineer/Analyst collaborates with clients, data scientists, and software engineers to develop and optimize data pipelines, ETL processes, and database structures that support IRS…
Added 6 mins ago
Associate Engineer - PC
AZEK Company
Wilmington, OH
…support manufacturing process design, reliability, operations, optimization, and continuous…
Added 6 mins ago
Associate Engineer - PC
AZEK Company
Wilmington, OH
…support manufacturing process design, reliability, operations, optimization, and continuous…
Added 13 mins ago
QA Engineer, Drug Substance Manufacturing
Fujifilm
Holly Springs, NC
The Engineer, QA Drug Substance Manufacturing is responsible to help drive the Quality oversight for the direction of all Drug Substance Manufacturing (DSM) processes through to operational…
Added 13 mins ago
QA Engineer, Drug Substance Manufacturing
Fujifilm
Holly Springs, NC
The QA Engineer, Drug Substance Manufacturing (DSM) Operations is responsible to help drive the Quality oversight for the direction of all Drug Substance Manufacturing (DSM) processes…
Added 13 mins ago
Information Systems Engineer Manager 2
CORP-Corporate Office
United States
Northrop Grumman Aeronautics Systems has an opening for an Information Systems Engineer Manager 2 to join our team of qualified, diverse individuals within our Aircraft Solutions…
Added 16 mins ago
CH
Data Analyst
Cayuse Holdings
Bismarck, ND
- $100,000 to $110,000 USD / year
The Data Engineer/Analyst collaborates with clients, data scientists, and software engineers to develop and optimize data pipelines, ETL processes, and database structures that support IRS…
Added 16 mins ago
Senior AWS Cloud Engineer
Serco
Bismarck, ND
Looking for a Senior Cloud Engineer opportunity in an environment where you can make an impact every day? Discover this great opportunity with the Center for Medicare and Medicaid Services…
Added 16 mins ago
Senior Network Engineer
Cognizant
Bismarck, ND
Cognizant is seeking a Senior Network Engineer to design, implement, secure, operate, and optimize enterprise datacenter, campus, branch, wireless, and hybrid-cloud networks. This role…
Added 17 mins ago
Senior Design Manager (Data Center)
Jacobs
Fargo, ND
…technology cooling water interfaces, leak detection, and thermal…
Added 17 mins ago
Senior Telecom Engineer
Maximus
Bismarck, ND
Maximus is seeking an experienced Senior Telecom Engineer to lead the design, implementation, and optimization of Genesys Cloud solutions. This role is responsible for executing large-scale…
Added 18 mins ago
G
Engineering Manager
GPAC
Chatham, IL
- $100,000 to $135,000 USD / year
This is an excellent opportunity for a Senior Manufacturing Engineer, Lead Engineer, or Engineering Supervisor ready to take the next step into a high-impact Engineering Manager role.
Added 18 mins ago
G
Engineering Manager
GPAC
Lincoln, IL
- $100,000 to $135,000 USD / year
This is an excellent opportunity for a Senior Manufacturing Engineer, Lead Engineer, or Engineering Supervisor ready to take the next step into a high-impact Engineering Manager role.
Added 22 mins ago
Distinguished Engineer, AI Security Systems (Remote)
CrowdStrike
Tallahassee, FL
We are seeking a Distinguished Engineer who has built AI and security systems that operate in the real world and stop real adversaries.
Added 23 mins ago
EG
Senior Infrastructure Engineer
Eliassen Group
Orlando, FL
- $84 to $86 USD / hour
Our client seeks a Senior Infrastructure Engineer to support and evolve cloud infrastructure powering analytics platforms, optimization engines, and machine learning workloads. The role…
Added 25 mins ago
V
Security Operations Center Operator - Weekends
VITALANT
United States
- $23 to $25 USD / hour
Added 31 mins ago
J
Support Operations Engineer
Jobgether
Remote, United States
- $100,000 to $130,000 CAD / year
We are seeking a hands-on Support Systems Engineer to take ownership of the systems and workflows that power a growing customer support organization. In this remote role, you will lead the…
Added 37 mins ago
Senior SIP Engineer
Telegent, Inc.
Pleasant Grove, UT
Token optimization, plan mode, and structured approaches like SDD , TDD , SpecKit, and role-based development
Added 41 mins ago
Software Engineer II - TeraWave
Blue Origin Personnel, LLC
Cupertino, CA
We are seeking a Software Engineer to join our PoP development team which is the high-performance, fast-path software that powers the TeraWave Point of Presence (PoP).
Added 48 mins ago
H
Advanced Chemical Engineer- Fired Heaters
Honeywell
Rosemont, IL · Houston, TX
- $104,000 to $149,000 USD / year
Provide expert-level equipment selection, design review and optimization for new units, revamps and debottlenecking efforts.
Added 50 mins ago
G
Maintenance Engineer
GPAC
Elk River, MN
- $34 to $41 USD / hour
A global leader in food processing technology is seeking a Maintenance Engineer to join its technical applications team.
Added 50 mins ago
G
Maintenance Engineer
GPAC
Eden Prairie, MN
- $34 to $41 USD / hour
A global leader in food processing technology is seeking a Maintenance Engineer to join its technical applications team.

Select Minds LLC

Senior AI Tools Testing Engineer

Dallas, TX, United States

$120,000 to $140,000 USD / year

Apply About employer

Applications open in a new tab.

Job details

Pay: $120,000 to $140,000 USD / year
Posting language: English
Posted: Nov 5, 2026

Original job description

Job Title: Senior / Lead AI Tools & Testing Engineer (Agentic AI & LLM Evaluation) Location: Dallas, TX (Hybrid) Duration: Long‑Term Experience:10 + Years

We seek a senior/lead AI engineer with hands‑on experience building, orchestrating, monitoring, and evaluating production‑grade Generative AI and Agentic AI applications. The ideal candidate is expert in Amazon Bedrock, AgentCore, LangGraph, LangChain, RAG, LLM observability, and evaluation frameworks. This role focuses on creating reliable, scalable, and production‑ready AI applications, emphasizing agent orchestration, AI testing, LLM evaluation, and continuous quality monitoring.

The candidate will collaborate with engineering, QA, and platform teams to establish best practices for building, testing, monitoring, and improving AI‑powered applications.

Key Responsibilities

Agentic AI & GenAI Engineering

Design, develop, and deploy production‑grade agentic and generative AI applications.
Build and orchestrate AI agents using LangGraph, LangChain, Amazon Bedrock, and AgentCore.
Implement multi‑agent workflows, tool calling, state management, memory, and orchestration.
Create scalable LLM‑powered applications and reusable AI components.

RAG & LLM Application Development

Design and implement Retrieval‑Augmented Generation (RAG) solutions.
Develop pipelines for document ingestion, chunking, embedding, retrieval, reranking, and context management.
Optimize retrieval accuracy, response quality, latency, and cost.
Apply grounding, hallucination reduction, and context‑relevance techniques.

LLM Evaluation & AI Testing

Build evaluation frameworks for LLM‑based applications and agentic workflows.
Automate evaluation with DeepEval and other LLM evaluation tools.
Use LLM‑as‑a‑Judge methods to assess response quality, relevance, accuracy, faithfulness, and reasoning.
Create evaluation datasets, test scenarios, benchmarks, and regression‑testing pipelines.
Evaluate agent behavior, tool execution, workflow accuracy, and response consistency.
Identify failure patterns and continuously improve AI application quality.

AI Observability & Monitoring

Implement tracing, monitoring, and observability with OpenTelemetry, LangSmith, and Langfuse.
Track LLM requests, agent execution flows, tool calls, latency, token consumption, errors, and performance.
Establish debugging mechanisms for complex multi‑step agentic workflows.
Develop monitoring practices to detect performance degradation and operational issues.

AWS AI Platform & Production Engineering

Deploy AI applications using Amazon Bedrock and Bedrock AgentCore.
Integrate foundation models, agent frameworks, APIs, and enterprise data sources.
Apply production‑readiness practices covering scalability, reliability, security, and performance.
Support CI/CD integration and automated evaluation of AI applications.
Optimize model selection, inference costs, latency, and resource utilization.

Technical Leadership & Collaboration

Provide technical guidance to engineering and QA teams.
Establish reusable frameworks, implementation patterns, and best practices.
Collaborate with architecture, DevOps, QA, and engineering teams.
Participate in design discussions, code reviews, proof‑of‑concept development, and implementation.
Mentor engineers and help teams adopt effective AI development and testing practices.

Required Technical Skills

Agentic AI & Orchestration

LangGraph
LangChain
Amazon Bedrock
Amazon Bedrock AgentCore
AI agents, multi‑agent orchestration, tool calling, workflow management

Generative AI & RAG

Large Language Models (LLMs)
Retrieval‑Augmented Generation (RAG)
Embeddings and vector search
Prompt engineering and context management
LLM application development

LLM Evaluation & Testing

DeepEval
LLM‑as‑a‑Judge techniques
Evaluation metrics and benchmarking
Automated evaluation and regression testing
Assessment of hallucination, faithfulness, relevance, and response quality

AI Observability & Monitoring

OpenTelemetry
LangSmith
Langfuse
LLM tracing, debugging, and performance monitoring

Production Engineering

Deploying and operating production‑grade GenAI applications
API integration and backend engineering
CI/CD and automated testing integration
Scalability, reliability, security, and performance optimization

Preferred Qualifications

8+ years of software engineering or IT experience.
Hands‑on experience developing and deploying GenAI or Agentic AI applications.
Experience with enterprise‑scale AI implementations.
Familiarity with AWS cloud services and enterprise AI security practices.
Experience establishing AI quality standards and evaluation methodologies.
Strong problem‑solving, communication, and technical collaboration skills.

What Success Looks Like

Development and deployment of production‑grade agentic AI applications.
Reliable orchestration and execution of AI agents and workflows.
Effective LLM evaluation and automated AI testing frameworks.
Improved AI application accuracy, reliability, observability, and performance.
Reusable engineering practices for building and maintaining enterprise GenAI solutions.

Important: This is a hands‑on engineering position. Candidates with practical experience building and evaluating AI applications are preferred over those whose experience is limited to AI strategy, architecture documentation, conventional QA automation, or using AI coding assistants.

Flexible work from home options available.

Apply About employer

More about this job

Skills

LangChain
Workflow
Debugging
AWS
CI/CD
Test automation
Problem solving
Communication skills
Collaboration

Report this job