Find work in the United States.

A simpler way to find your next job.

Jobs in the United States

25+ jobs

Save this search

Added 3 mins ago
O
Lead Principal Software Engineer, Core Infrastructure
Oracle
Nashville, TN
- $146,300 to $306,400 USD / year
…solver, a distributed systems generalist and/or skilled Linux engineer with Systems triage experience able to dive deep into any part of the stack and low-level systems to design broad…
Added 3 mins ago
O
Lead Principal Platform Software Engineer
Oracle
Seattle, WA · Santa Clara, CA
- $113,600 to $306,400 USD / year
We are seeking a Lead Principal Platform Software Engineer to lead the architecture and delivery of critical platform systems and initiatives across teams. This is a hands-on technical…
Added 4 mins ago
O
Associate Support Engineer
Oracle
Montpelier, VT
- $21.3 to $45.05 USD / hour
Added 4 mins ago
O
Senior Core Infrastructure Engineer - Nashville, TN
Oracle
Montpelier, VT
- $79,200 to $209,500 USD / year
Added 4 mins ago
Senior Software Engineer (Institutional, Financing)
Coinbase
Montpelier, VT
As a Senior Software Engineer on the Financing Engineering team within the Institutional group, you'll design, build, and scale the systems that safely manage billions of dollars in loans…
Added 5 mins ago
O
Commercial Technical Program Manager
Oracle
Montpelier, VT
- $126,200 to $264,100 USD / year
Added 5 mins ago
Distinguished Engineer, AI Security Systems (Remote)
CrowdStrike
Montpelier, VT
We are seeking a Distinguished Engineer who has built AI and security systems that operate in the real world and stop real adversaries.
Added 5 mins ago
O
Senior Database Site Reliability Engineer – Oracle Database and GoldenGate (IC3)
Oracle
Montpelier, VT
- $81,100 to $187,000 USD / year
Oracle Health Database Services is seeking a highly experienced, Site Reliability Engineer to lead database reliability engineering across mission-critical healthcare platforms.
Added 5 mins ago
AT
Network Infrastructure Engineer II
Akamai Technologies, Inc.
Montpelier, VT
- $83,200 to $149,800 USD / year
As a Network Infrastructure Engineer II, you will oversee operations in our global data centers. You will monitor and maintain our data centers, networks, and servers worldwide. You will…
Added 5 mins ago
Software Engineer
Teradata
Montpelier, VT
This position is for a Software Engineer in the POD Services team within the TD Factory team. The position involves development, integration, functional and performance testing, and fault…
Added 5 mins ago
Software Engineer, Infrastructure - Central Growth
Meta
Montpelier, VT
Software Engineer, Infrastructure - Central Growth Responsibilities:
Added 5 mins ago
AI Automation Architect - Manufacturing and MES processes- Remote
Cognizant
Montpelier, VT
…fundamentally connect people with processes. Within Cognizant IOT, we engineer industry-aligned, IoT-enabled products that merge industry needs with human drivers. Our intelligent products…
Added 5 mins ago
EG
Workday Adaptive Planning Engineer
Eliassen Group
Montpelier, VT
- $65 to $75 USD / hour
Our client is seeking a Workday Adaptive Planning Engineer to design, build, and maintain financial planning and forecasting solutions. The role partners with FP&A, Accounting, and Finance…
Added 5 mins ago
EG
Quality Assurance Engineer - PHP
Eliassen Group
Montpelier, VT
- $40 to $50 USD / hour
Our client seeks a results-driven Senior Quality Assurance Engineer to support Policyholder Portal applications. The role requires manual, automation, mobile, API, and database testing…
Added 10 mins ago
Tosca Test Automation Lead
QualityAI
Santa Clara, CA
Senior Test Automation Engineer with 10+ years of experience in automation framework design, development, and maintenance. The ideal candidate has strong expertise in Tricentis Tosca,…
Added 16 mins ago
ZT
Quality Control Laboratory Technician, Support I
Zebra Technologies
Morris Plains, NJ
- $23 USD / hour
Monitor and report test status of various production runs to QC Manager, Manufacturing Manager, QC Supervisor, Technology, or Process Engineer.
Added 16 mins ago
Lead AI Engineer
Guardian Life
Holmdel, NJ
The employee serves as a Lead AI Engineer responsible for architecting, developing, deploying, and supporting enterprise-scale Agentic AI solutions.
Added 16 mins ago
Senior Software Engineer (Institutional, Financing)
Coinbase
Trenton, NJ
As a Senior Software Engineer on the Financing Engineering team within the Institutional group, you'll design, build, and scale the systems that safely manage billions of dollars in loans…
Added 16 mins ago
Distinguished Engineer, AI Security Systems (Remote)
CrowdStrike
Trenton, NJ
We are seeking a Distinguished Engineer who has built AI and security systems that operate in the real world and stop real adversaries.
Added 21 mins ago
WI
(USA) Principal, Software Engineer
Walmart Inc.
United States
- $143,000 to $286,000 USD / year
The (USA) Principal, Software Engineer leads the design and delivery of scalable platform capabilities, driving innovation in cloud-native architectures and AI/ML integration. This role…
Added 21 mins ago
WI
Software Engineer II | Bentonville, Arkansas
Walmart Inc.
United States
- $80,000 to $155,000 USD / year
As a Software Engineer II, you will deliver high-quality software solutions by collaborating with cross-functional teams to design, develop, and maintain scalable applications and services.…
Added 21 mins ago
TT
Forward Deployed Engineer
True Talent Solutions Partners
San Francisco, CA
- $160,000 to $275,000 USD / year
Added 25 mins ago
Senior Software Engineer II
Samsara
Remote, United States
We are looking for a product-minded senior engineer who can own ambiguous problems end to end, make practical tradeoffs across surfaces, and ship work that holds up in the field. You will…
Added 25 mins ago
IG
Software Quality Engineer
Insight Global
Sunset Hills, MO
- $35 to $40 USD / hour
We are seeking a Junior Software Quality Engineer with strong test automation experience and prior hands-on experience applying AI-assisted testing practices. In this role, you will help…
Added 27 mins ago
Software Engineer, Sensors and System Software (C++, Linux)
Avride
Austin, TX
We are seeking a Software Engineer responsible for integrating the diverse array of sensors and various embedded hardware into our software pipelines. In this critical role, you will be…

Select Minds LLC

Senior AI Tools Testing Engineer

Dallas, TX, United States

$120,000 to $140,000 USD / year

Apply About employer

Applications open in a new tab.

Job details

Pay: $120,000 to $140,000 USD / year
Posting language: English
Posted: Nov 5, 2026

Original job description

Job Title: Senior / Lead AI Tools & Testing Engineer (Agentic AI & LLM Evaluation) Location: Dallas, TX (Hybrid) Duration: Long‑Term Experience:10 + Years

We seek a senior/lead AI engineer with hands‑on experience building, orchestrating, monitoring, and evaluating production‑grade Generative AI and Agentic AI applications. The ideal candidate is expert in Amazon Bedrock, AgentCore, LangGraph, LangChain, RAG, LLM observability, and evaluation frameworks. This role focuses on creating reliable, scalable, and production‑ready AI applications, emphasizing agent orchestration, AI testing, LLM evaluation, and continuous quality monitoring.

The candidate will collaborate with engineering, QA, and platform teams to establish best practices for building, testing, monitoring, and improving AI‑powered applications.

Key Responsibilities

Agentic AI & GenAI Engineering

Design, develop, and deploy production‑grade agentic and generative AI applications.
Build and orchestrate AI agents using LangGraph, LangChain, Amazon Bedrock, and AgentCore.
Implement multi‑agent workflows, tool calling, state management, memory, and orchestration.
Create scalable LLM‑powered applications and reusable AI components.

RAG & LLM Application Development

Design and implement Retrieval‑Augmented Generation (RAG) solutions.
Develop pipelines for document ingestion, chunking, embedding, retrieval, reranking, and context management.
Optimize retrieval accuracy, response quality, latency, and cost.
Apply grounding, hallucination reduction, and context‑relevance techniques.

LLM Evaluation & AI Testing

Build evaluation frameworks for LLM‑based applications and agentic workflows.
Automate evaluation with DeepEval and other LLM evaluation tools.
Use LLM‑as‑a‑Judge methods to assess response quality, relevance, accuracy, faithfulness, and reasoning.
Create evaluation datasets, test scenarios, benchmarks, and regression‑testing pipelines.
Evaluate agent behavior, tool execution, workflow accuracy, and response consistency.
Identify failure patterns and continuously improve AI application quality.

AI Observability & Monitoring

Implement tracing, monitoring, and observability with OpenTelemetry, LangSmith, and Langfuse.
Track LLM requests, agent execution flows, tool calls, latency, token consumption, errors, and performance.
Establish debugging mechanisms for complex multi‑step agentic workflows.
Develop monitoring practices to detect performance degradation and operational issues.

AWS AI Platform & Production Engineering

Deploy AI applications using Amazon Bedrock and Bedrock AgentCore.
Integrate foundation models, agent frameworks, APIs, and enterprise data sources.
Apply production‑readiness practices covering scalability, reliability, security, and performance.
Support CI/CD integration and automated evaluation of AI applications.
Optimize model selection, inference costs, latency, and resource utilization.

Technical Leadership & Collaboration

Provide technical guidance to engineering and QA teams.
Establish reusable frameworks, implementation patterns, and best practices.
Collaborate with architecture, DevOps, QA, and engineering teams.
Participate in design discussions, code reviews, proof‑of‑concept development, and implementation.
Mentor engineers and help teams adopt effective AI development and testing practices.

Required Technical Skills

Agentic AI & Orchestration

LangGraph
LangChain
Amazon Bedrock
Amazon Bedrock AgentCore
AI agents, multi‑agent orchestration, tool calling, workflow management

Generative AI & RAG

Large Language Models (LLMs)
Retrieval‑Augmented Generation (RAG)
Embeddings and vector search
Prompt engineering and context management
LLM application development

LLM Evaluation & Testing

DeepEval
LLM‑as‑a‑Judge techniques
Evaluation metrics and benchmarking
Automated evaluation and regression testing
Assessment of hallucination, faithfulness, relevance, and response quality

AI Observability & Monitoring

OpenTelemetry
LangSmith
Langfuse
LLM tracing, debugging, and performance monitoring

Production Engineering

Deploying and operating production‑grade GenAI applications
API integration and backend engineering
CI/CD and automated testing integration
Scalability, reliability, security, and performance optimization

Preferred Qualifications

8+ years of software engineering or IT experience.
Hands‑on experience developing and deploying GenAI or Agentic AI applications.
Experience with enterprise‑scale AI implementations.
Familiarity with AWS cloud services and enterprise AI security practices.
Experience establishing AI quality standards and evaluation methodologies.
Strong problem‑solving, communication, and technical collaboration skills.

What Success Looks Like

Development and deployment of production‑grade agentic AI applications.
Reliable orchestration and execution of AI agents and workflows.
Effective LLM evaluation and automated AI testing frameworks.
Improved AI application accuracy, reliability, observability, and performance.
Reusable engineering practices for building and maintaining enterprise GenAI solutions.

Important: This is a hands‑on engineering position. Candidates with practical experience building and evaluating AI applications are preferred over those whose experience is limited to AI strategy, architecture documentation, conventional QA automation, or using AI coding assistants.

Flexible work from home options available.

Apply About employer

More about this job

Skills

LangChain
Workflow
Debugging
AWS
CI/CD
Test automation
Problem solving
Communication skills
Collaboration

Report this job