Find work in the United States.

A simpler way to find your next job.

Jobs in the United States

25+ jobs

Save this search

Added just now
Distinguished Engineer, AI Security Systems (Remote)
CrowdStrike
Montpelier, VT
We are seeking a Distinguished Engineer who has built AI and security systems that operate in the real world and stop real adversaries.
Added just now
Software Engineer II
Amentum
Montpelier, VT
The Software Engineer II will perform full stack development with C# program, ReactJS, MVC, Linq, and rest API development in NET Core platform. This position is US remote teleworking…
Added just now
Platform Engineer - Remote
SitusAMC
Montpelier, VT
Develop and maintain the in-house CI/CD automation pipeline as an individual contributor on the Cloud development platform team.
Added just now
GA
Sr Manufacturing Engineer 2 – Sourced Hardware
GE Aerospace
Montpelier, VT
- $136,000 to $181,000 USD / year
The Senior Manufacturing Engineer will manage and coordinate delivery projects within the Supplier Performance & Execution Sourcing Section. You will organize and coordinate…
Added just now
GA
Sr Manufacturing Engineer - Sourced Hardware
GE Aerospace
Montpelier, VT
- $116,000 to $155,000 USD / year
The Senior Manufacturing Engineer will manage and coordinate delivery projects within the Supplier Performance & Execution Sourcing Section. You will organize and coordinate…
Added just now
Software Engineer, Infrastructure - Central Growth
Meta
Montpelier, VT
Software Engineer, Infrastructure - Central Growth Responsibilities:
Added just now
Senior Member of Technical Staff (SMTS) – Quantum Materials & Thin Films Engineering
Global Foundries
Essex Junction, VT
GlobalFoundries Quantum Technology Solutions (QTS) is seeking an experienced materials engineer to drive the development, characterization, and integration of advanced materials enabling…
Added just now
Mainframe Technical Engineer
Highmark Health
Montpelier, VT
This job is an integral member of one of our teams responsible for supporting the design, development, and maintenance of our Mainframe technical solutions, ensuring their efficient and…
Added just now
AI Automation Architect - Manufacturing and MES processes- Remote
Cognizant
Montpelier, VT
…fundamentally connect people with processes. Within Cognizant IOT, we engineer industry-aligned, IoT-enabled products that merge industry needs with human drivers. Our intelligent products…
Added just now
GA
Sr Manufacturing Engineer - Sourced Hardware
GE Aerospace
Montpelier, VT
- $136,000 to $181,000 USD / year
The Senior Manufacturing Engineer will manage and coordinate delivery projects within the Supplier Performance & Execution Sourcing Section. You will organize and coordinate…
Added just now
GA
Sr Manufacturing Engineer - Sourced Hardware
GE Aerospace
Montpelier, VT
- $116,000 to $155,000 USD / year
The Senior Manufacturing Engineer will manage and coordinate delivery projects within the Supplier Performance & Execution Sourcing Section. You will organize and coordinate…
Added 1 min ago
G
Structures Group Manager PE - Transportation Bridges
GPAC
Lone Tree, CO
- $145,000 to $175,000 USD / year
…practice in the Denver area. They're looking for a senior bridge engineer ready to step into group leadership, with a structured hand-off and mentorship from the current group…
Added 1 min ago
G
Structures Group Manager PE - Transportation Bridges
GPAC
Littleton, CO
- $145,000 to $175,000 USD / year
…practice in the Denver area. They're looking for a senior bridge engineer ready to step into group leadership, with a structured hand-off and mentorship from the current group…
Added 1 min ago
G
Structures Group Manager PE - Transportation Bridges
GPAC
Parker, CO
- $145,000 to $175,000 USD / year
…practice in the Denver area. They're looking for a senior bridge engineer ready to step into group leadership, with a structured hand-off and mentorship from the current group…
Added 1 min ago
G
Structures Group Manager PE - Transportation Bridges
GPAC
Aurora, CO
- $145,000 to $175,000 USD / year
…practice in the Denver area. They're looking for a senior bridge engineer ready to step into group leadership, with a structured hand-off and mentorship from the current group…
Added 1 min ago
G
Structures Group Manager PE - Transportation Bridges
GPAC
Centennial, CO
- $145,000 to $175,000 USD / year
…practice in the Denver area. They're looking for a senior bridge engineer ready to step into group leadership, with a structured hand-off and mentorship from the current group…
Added 2 mins ago
Civil Engineer
GPAC
Poughkeepsie, NY
A well-established, growing engineering firm in your area is seeking 3+ year Civil Engineer to join their team. They have a work hard/play hard mentality, full benefits, and exceptionally…
Added 2 mins ago
Civil Engineer
GPAC
Raleigh, NC
A well-established, growing engineering firm in your area is seeking 3+ year Civil Engineer to join their team. They have a work hard/play hard mentality, full benefits, and exceptionally…
Added 2 mins ago
Civil Engineer
GPAC
Raleigh, NC
A well-established, growing engineering firm in your area is seeking 3+ year Civil Engineer to join their team. They have a work hard/play hard mentality, full benefits, and exceptionally…
Added 3 mins ago
Mechatronics Engineer III
JPL/NASA
Pasadena, CA
You will be working as a Mechatronics Engineer within the Mechanical Engineering, Integration, and Test Section, which leads the design, analysis, development, integration, and test of…
Added 3 mins ago
G
Municipal Engineer, PE - Mid/Senior Level
GPAC
Southgate, MI
- $90,000 to $130,000 USD / year
Our client, a respected civil engineering firm in Southeast Michigan, is seeking a Municipal Engineer at the mid or senior level to deliver infrastructure projects for local communities.
Added 3 mins ago
Mechanical Engineering Specialist
Lynntech, Inc.
College Station, TX
The engineer will execute technical work as part of multidisciplinary project teams and will have opportunities to develop leadership skills through increasing technical responsibility,…
Added 4 mins ago
Civil Engineer
GPAC
Raleigh, NC
A well-established, growing engineering firm in your area is seeking 3+ year Civil Engineer to join their team. They have a work hard/play hard mentality, full benefits, and exceptionally…
Added 4 mins ago
Civil Engineer
GPAC
Raleigh, NC
A well-established, growing engineering firm in your area is seeking 3+ year Civil Engineer to join their team. They have a work hard/play hard mentality, full benefits, and exceptionally…
Added 4 mins ago
H
Lead System Design Engineer
HighRes
Beverly, MA
- $100,000 to $120,000 USD / year
The Lead System Design Engineer (SDE) will be the technical expert for customer facing projects post purchase order. They will be the gating resource for handing a project with finalized…

Select Minds LLC

Senior AI Tools Testing Engineer

Dallas, TX, United States

$120,000 to $140,000 USD / year

Apply About employer

Applications open in a new tab.

Job details

Pay: $120,000 to $140,000 USD / year
Posting language: English
Posted: Nov 5, 2026

Original job description

Job Title: Senior / Lead AI Tools & Testing Engineer (Agentic AI & LLM Evaluation) Location: Dallas, TX (Hybrid) Duration: Long‑Term Experience:10 + Years

We seek a senior/lead AI engineer with hands‑on experience building, orchestrating, monitoring, and evaluating production‑grade Generative AI and Agentic AI applications. The ideal candidate is expert in Amazon Bedrock, AgentCore, LangGraph, LangChain, RAG, LLM observability, and evaluation frameworks. This role focuses on creating reliable, scalable, and production‑ready AI applications, emphasizing agent orchestration, AI testing, LLM evaluation, and continuous quality monitoring.

The candidate will collaborate with engineering, QA, and platform teams to establish best practices for building, testing, monitoring, and improving AI‑powered applications.

Key Responsibilities

Agentic AI & GenAI Engineering

Design, develop, and deploy production‑grade agentic and generative AI applications.
Build and orchestrate AI agents using LangGraph, LangChain, Amazon Bedrock, and AgentCore.
Implement multi‑agent workflows, tool calling, state management, memory, and orchestration.
Create scalable LLM‑powered applications and reusable AI components.

RAG & LLM Application Development

Design and implement Retrieval‑Augmented Generation (RAG) solutions.
Develop pipelines for document ingestion, chunking, embedding, retrieval, reranking, and context management.
Optimize retrieval accuracy, response quality, latency, and cost.
Apply grounding, hallucination reduction, and context‑relevance techniques.

LLM Evaluation & AI Testing

Build evaluation frameworks for LLM‑based applications and agentic workflows.
Automate evaluation with DeepEval and other LLM evaluation tools.
Use LLM‑as‑a‑Judge methods to assess response quality, relevance, accuracy, faithfulness, and reasoning.
Create evaluation datasets, test scenarios, benchmarks, and regression‑testing pipelines.
Evaluate agent behavior, tool execution, workflow accuracy, and response consistency.
Identify failure patterns and continuously improve AI application quality.

AI Observability & Monitoring

Implement tracing, monitoring, and observability with OpenTelemetry, LangSmith, and Langfuse.
Track LLM requests, agent execution flows, tool calls, latency, token consumption, errors, and performance.
Establish debugging mechanisms for complex multi‑step agentic workflows.
Develop monitoring practices to detect performance degradation and operational issues.

AWS AI Platform & Production Engineering

Deploy AI applications using Amazon Bedrock and Bedrock AgentCore.
Integrate foundation models, agent frameworks, APIs, and enterprise data sources.
Apply production‑readiness practices covering scalability, reliability, security, and performance.
Support CI/CD integration and automated evaluation of AI applications.
Optimize model selection, inference costs, latency, and resource utilization.

Technical Leadership & Collaboration

Provide technical guidance to engineering and QA teams.
Establish reusable frameworks, implementation patterns, and best practices.
Collaborate with architecture, DevOps, QA, and engineering teams.
Participate in design discussions, code reviews, proof‑of‑concept development, and implementation.
Mentor engineers and help teams adopt effective AI development and testing practices.

Required Technical Skills

Agentic AI & Orchestration

LangGraph
LangChain
Amazon Bedrock
Amazon Bedrock AgentCore
AI agents, multi‑agent orchestration, tool calling, workflow management

Generative AI & RAG

Large Language Models (LLMs)
Retrieval‑Augmented Generation (RAG)
Embeddings and vector search
Prompt engineering and context management
LLM application development

LLM Evaluation & Testing

DeepEval
LLM‑as‑a‑Judge techniques
Evaluation metrics and benchmarking
Automated evaluation and regression testing
Assessment of hallucination, faithfulness, relevance, and response quality

AI Observability & Monitoring

OpenTelemetry
LangSmith
Langfuse
LLM tracing, debugging, and performance monitoring

Production Engineering

Deploying and operating production‑grade GenAI applications
API integration and backend engineering
CI/CD and automated testing integration
Scalability, reliability, security, and performance optimization

Preferred Qualifications

8+ years of software engineering or IT experience.
Hands‑on experience developing and deploying GenAI or Agentic AI applications.
Experience with enterprise‑scale AI implementations.
Familiarity with AWS cloud services and enterprise AI security practices.
Experience establishing AI quality standards and evaluation methodologies.
Strong problem‑solving, communication, and technical collaboration skills.

What Success Looks Like

Development and deployment of production‑grade agentic AI applications.
Reliable orchestration and execution of AI agents and workflows.
Effective LLM evaluation and automated AI testing frameworks.
Improved AI application accuracy, reliability, observability, and performance.
Reusable engineering practices for building and maintaining enterprise GenAI solutions.

Important: This is a hands‑on engineering position. Candidates with practical experience building and evaluating AI applications are preferred over those whose experience is limited to AI strategy, architecture documentation, conventional QA automation, or using AI coding assistants.

Flexible work from home options available.

Apply About employer

More about this job

Skills

LangChain
Workflow
Debugging
AWS
CI/CD
Test automation
Problem solving
Communication skills
Collaboration

Report this job