Find work in the United States.

A simpler way to find your next job.

Jobs in the United States

25+ jobs

Save this search

Added just now
Network Engineer, Deployment & Support
Meta
Jefferson City, MO
Network Engineer, Deployment & Support Responsibilities:
Added 2 mins ago
Project Engineer
CF Industries Employee Services LLC
Donaldsonville, LA
…work includes, but may not be limited to: facilitating scope development of capital projects, compiling estimates based on developed scope, preparing Authorization For Expenditures (AFEs)…
Added 2 mins ago
NJ
Information Technology Analyst 3 - Engineering
New Jersey Courts
Trenton, NJ
- $94,668.29 to $133,073.24 USD / year
…the Information Security Unit, working primarily as the lead engineer and technical expert for the unit’s Engineering…
Added 2 mins ago
G
Civil Engineer
GPAC
Atlanta, GA
- $60,000 to $90,000 USD / year
A well-established firm in Columbia, SC is seeking a Civil Design Engineer to join their dynamic team. The ideal candidate will have experience in site civil land development and possess a…
Added 2 mins ago
G
Civil Engineer
GPAC
Denver, CO
- $60,000 to $90,000 USD / year
A well-established firm in Columbia, SC is seeking a Civil Design Engineer to join their dynamic team. The ideal candidate will have experience in site civil land development and possess a…
Added 3 mins ago
Engineer
The Metropolitan Water District of Southern California
Los Angeles, CA
License in good standing as a California Professional Engineer (required at time of application).
Added 3 mins ago
Sr Engineer, Software/Information Platform
Cardinal Health
Jefferson City, MO
Software & Information Platforms manages the technical configuration, design, administration, development, implementation and support of application and information frameworks that the…
Added 3 mins ago
Senior Lead Engineer
PWC US
IL-Chicago, IL · NC-Raleigh, NC · NC-Charlotte, NC · OH-Cincinnati, OH · OH-Cleveland, OH · SC-Columbia, SC · OH-Columbus, OH · TX-Dallas, TX · VA-Richmond, VA · MD-Baltimore, MD · NY-Rochester, NY · IL-Rosemont, IL · CA-Sacramento, CA · MA-Boston, MA · MO-St. Louis, MO · UT-Salt Lake City, UT · TX-San Antonio, TX · CA-San Diego, CA · CA-San Francisco, CA · CA-Silicon Valley, CA · WA-Seattle, WA · SC-Spartanburg, SC · NY-Albany, NY · CT-Stamford, CT · FL-Tampa, FL · GA-Atlanta, GA · OH-Toledo, OH · OK-Tulsa, OK · TX-Austin, TX · DC-Washington, DC · CO-Denver, CO · IA-Des Moines, IA · FL-Boca Raton, FL · MI-Detroit, MI · NY-Buffalo, NY · NJ-Florham Park, NJ · TX-Fort Worth, TX · NC-Greensboro, NC · CT-Hartford, CT · TX-Houston, TX · IN-Indianapolis, IN · MI-Grand Rapids, MI · CA-Irvine, CA · FL-Jacksonville, FL · MO-Kansas City, MO · NV-Las Vegas, NV · AR-Little Rock, AR · CA-Los Angeles, CA · KY-Louisville, KY · AL-Birmingham, AL · AR-Fayetteville, AR · NY-Melville, NY · FL-Miami, FL · WI-Milwaukee, WI · MN-Minneapolis, MN · VT-Montpelier, VT · TN-Nashville, TN · LA-New Orleans, LA · NY-New York, NY · OK-Oklahoma City, OK · FL-Orlando, FL · PA-Philadelphia, PA · AZ-Phoenix, AZ · PA-Pittsburgh, PA · OR-Portland, OR
As a Senior Lead Engineer you will lead custom application development and testing efforts that help clients turn business needs into scalable software solutions across cloud-native and…
Added 3 mins ago
Senior Engineer
PWC US
IL-Chicago, IL · NC-Raleigh, NC · NC-Charlotte, NC · OH-Cincinnati, OH · OH-Cleveland, OH · SC-Columbia, SC · OH-Columbus, OH · TX-Dallas, TX · VA-Richmond, VA · MD-Baltimore, MD · NY-Rochester, NY · IL-Rosemont, IL · CA-Sacramento, CA · MA-Boston, MA · MO-St. Louis, MO · UT-Salt Lake City, UT · TX-San Antonio, TX · CA-San Diego, CA · CA-San Francisco, CA · CA-Silicon Valley, CA · WA-Seattle, WA · SC-Spartanburg, SC · NY-Albany, NY · CT-Stamford, CT · FL-Tampa, FL · GA-Atlanta, GA · OH-Toledo, OH · OK-Tulsa, OK · TX-Austin, TX · DC-Washington, DC · CO-Denver, CO · IA-Des Moines, IA · FL-Boca Raton, FL · MI-Detroit, MI · NY-Buffalo, NY · NJ-Florham Park, NJ · TX-Fort Worth, TX · NC-Greensboro, NC · CT-Hartford, CT · TX-Houston, TX · IN-Indianapolis, IN · MI-Grand Rapids, MI · CA-Irvine, CA · FL-Jacksonville, FL · MO-Kansas City, MO · NV-Las Vegas, NV · AR-Little Rock, AR · CA-Los Angeles, CA · KY-Louisville, KY · AL-Birmingham, AL · AR-Fayetteville, AR · NY-Melville, NY · FL-Miami, FL · WI-Milwaukee, WI · MN-Minneapolis, MN · VT-Montpelier, VT · TN-Nashville, TN · LA-New Orleans, LA · OK-Oklahoma City, OK · FL-Orlando, FL · PA-Philadelphia, PA · AZ-Phoenix, AZ · PA-Pittsburgh, PA · OR-Portland, OR
As a Senior Engineer you will help design, build, and test custom applications within our Advisory practice, bringing together software engineering, cloud-based solution delivery, and…
Added 3 mins ago
Lead Engineer
PWC US
IL-Chicago, IL · NC-Raleigh, NC · NC-Charlotte, NC · OH-Cincinnati, OH · OH-Cleveland, OH · SC-Columbia, SC · OH-Columbus, OH · TX-Dallas, TX · VA-Richmond, VA · MD-Baltimore, MD · NY-Rochester, NY · IL-Rosemont, IL · CA-Sacramento, CA · MA-Boston, MA · MO-St. Louis, MO · UT-Salt Lake City, UT · TX-San Antonio, TX · CA-San Diego, CA · CA-San Francisco, CA · CA-Silicon Valley, CA · WA-Seattle, WA · SC-Spartanburg, SC · NY-Albany, NY · CT-Stamford, CT · FL-Tampa, FL · GA-Atlanta, GA · OH-Toledo, OH · OK-Tulsa, OK · TX-Austin, TX · DC-Washington, DC · CO-Denver, CO · IA-Des Moines, IA · FL-Boca Raton, FL · MI-Detroit, MI · NY-Buffalo, NY · NJ-Florham Park, NJ · TX-Fort Worth, TX · NC-Greensboro, NC · CT-Hartford, CT · TX-Houston, TX · IN-Indianapolis, IN · MI-Grand Rapids, MI · CA-Irvine, CA · FL-Jacksonville, FL · MO-Kansas City, MO · NV-Las Vegas, NV · AR-Little Rock, AR · CA-Los Angeles, CA · KY-Louisville, KY · AL-Birmingham, AL · AR-Fayetteville, AR · NY-Melville, NY · FL-Miami, FL · WI-Milwaukee, WI · MN-Minneapolis, MN · VT-Montpelier, VT · TN-Nashville, TN · LA-New Orleans, LA · NY-New York, NY · OK-Oklahoma City, OK · FL-Orlando, FL · PA-Philadelphia, PA · AZ-Phoenix, AZ · PA-Pittsburgh, PA · OR-Portland, OR
As a Lead Engineer you will guide the design, build, testing, and modernization of custom applications across cloud-native and on-premise environments, using modern engineering approaches,…
Added 4 mins ago
Product Lifecycle Engineer, Generators, Data Center Electrical Products and Services
Amazon Data Services, Inc.
Austin, TX · Seattle, WA · Atlanta, GA · Columbus, OH · Herndon, VA · Chicago, IL
As a Global AWS Product Lifecycle Engineer, you will provide electrical infrastructure support to AWS data centers. You will collaborate with the product development team to gain deep…
Added 5 mins ago
Senior Fullstack Engineer, FanX AI
SeatGeek
NY
Mentor and grow web engineers across the organization as you remain on the cutting edge of AI feature development and tool use
Added 5 mins ago
Senior Fullstack Engineer, FanX AI
SeatGeek
Remote, United States
Mentor and grow web engineers across the organization as you remain on the cutting edge of AI feature development and tool use
Added 5 mins ago
DevOps Engineer 1
Qu Pos, Inc.
Remote, United States
Experience working in a product company or product type software development experience
Added 6 mins ago
VR
Sr. Platform Engineer - REMOTE
VWCU Re-Brand
Tucson, AZ
- $100,568 to $128,224.2 USD / year
The Senior Platform Engineer designs, implements, operates, and improves enterprise infrastructure across on-premises, cloud, and hybrid environments. This role supports Windows Server,…
Added 7 mins ago
Site Reliability Engineer (Starlink)
SpaceX
Redmond, WA · Hawthorne, CA · Palo Alto, CA
Engage throughout the whole software development lifecycle of services -- from inception to design, deployment, operation, and iterative refinement
Added 7 mins ago
Site Reliability Engineer (Starlink)
SpaceX
Hawthorne, CA · Palo Alto, CA · Redmond, WA
Engage throughout the whole software development lifecycle of services -- from inception to design, deployment, operation, and iterative refinement
Added 7 mins ago
Site Reliability Engineer (Starlink)
SpaceX
Palo Alto, CA · Hawthorne, CA · Redmond, WA
Engage throughout the whole software development lifecycle of services -- from inception to design, deployment, operation, and iterative refinement
Added 7 mins ago
IT Network Infrastructure Engineer, Outside Plant
SpaceX
College Station, TX
Support BOM development, quantity take-offs, and basic loss-budget calculations.
Added 7 mins ago
Senior Electrical Engineer
Serco USA
Wright-Patterson AFB, OH · Andrews AFB, MD · Beavercreek, OH · Dayton, OH · Huber Heights, OH · Kettering, OH · Little Rock AFB, AR · London, OH · McConnell AFB, KS · Montgomery, OH · OH · Scott AFB, IL · Springfield, OH · Whiteman AFB, MO
If you want to work on cutting-edge DoD programs, Serco has a great opportunity for you! As a Senior Electrical Engineer, you will be supporting the Special Programs Division, DoD…
Added 7 mins ago
Fellow, Manufacturing Systems Engineer
Firestorm
United States
Lead cross-functional project teams through development, prove-out, and deployment, and mentor other engineers along the way.
Added 10 mins ago
Lead Software Engineer - Embedded
The Boeing Company
Remote, United States
High-Integrity Software Development: Deriving and allocating requirements with traceability throughout the software lifecycle. Designing solutions that meet requirements and defining…
Added 11 mins ago
Senior Automation & Controls Engineer - Food & Beverage (Atlanta)
Burns & McDonnell
Kansas City, MO
The Senior Automation & Controls Engineer leads and executes Automation and Controls Engineering design activities for minor to major scale Engineering, Procurement, and Construction (EPC)…
Added 11 mins ago
Software Engineer II
Western Governors University
Salt Lake City, UT
As a Software Engineer II, you will help research, design, develop, test, and implement applications and software systems that support WGU’s mission. You will contribute to an experienced…
Added 11 mins ago
Associate Software Engineer
Western Governors University
Salt Lake City, UT
Recent graduate or early-career software engineer with a strong foundation in software development principles.

Select Minds LLC

Senior AI Tools Testing Engineer

Dallas, TX, United States

$120,000 to $140,000 USD / year

Apply About employer

Applications open in a new tab.

Job details

Pay: $120,000 to $140,000 USD / year
Posting language: English
Posted: Nov 5, 2026

Original job description

Job Title: Senior / Lead AI Tools & Testing Engineer (Agentic AI & LLM Evaluation) Location: Dallas, TX (Hybrid) Duration: Long‑Term Experience:10 + Years

We seek a senior/lead AI engineer with hands‑on experience building, orchestrating, monitoring, and evaluating production‑grade Generative AI and Agentic AI applications. The ideal candidate is expert in Amazon Bedrock, AgentCore, LangGraph, LangChain, RAG, LLM observability, and evaluation frameworks. This role focuses on creating reliable, scalable, and production‑ready AI applications, emphasizing agent orchestration, AI testing, LLM evaluation, and continuous quality monitoring.

The candidate will collaborate with engineering, QA, and platform teams to establish best practices for building, testing, monitoring, and improving AI‑powered applications.

Key Responsibilities

Agentic AI & GenAI Engineering

Design, develop, and deploy production‑grade agentic and generative AI applications.
Build and orchestrate AI agents using LangGraph, LangChain, Amazon Bedrock, and AgentCore.
Implement multi‑agent workflows, tool calling, state management, memory, and orchestration.
Create scalable LLM‑powered applications and reusable AI components.

RAG & LLM Application Development

Design and implement Retrieval‑Augmented Generation (RAG) solutions.
Develop pipelines for document ingestion, chunking, embedding, retrieval, reranking, and context management.
Optimize retrieval accuracy, response quality, latency, and cost.
Apply grounding, hallucination reduction, and context‑relevance techniques.

LLM Evaluation & AI Testing

Build evaluation frameworks for LLM‑based applications and agentic workflows.
Automate evaluation with DeepEval and other LLM evaluation tools.
Use LLM‑as‑a‑Judge methods to assess response quality, relevance, accuracy, faithfulness, and reasoning.
Create evaluation datasets, test scenarios, benchmarks, and regression‑testing pipelines.
Evaluate agent behavior, tool execution, workflow accuracy, and response consistency.
Identify failure patterns and continuously improve AI application quality.

AI Observability & Monitoring

Implement tracing, monitoring, and observability with OpenTelemetry, LangSmith, and Langfuse.
Track LLM requests, agent execution flows, tool calls, latency, token consumption, errors, and performance.
Establish debugging mechanisms for complex multi‑step agentic workflows.
Develop monitoring practices to detect performance degradation and operational issues.

AWS AI Platform & Production Engineering

Deploy AI applications using Amazon Bedrock and Bedrock AgentCore.
Integrate foundation models, agent frameworks, APIs, and enterprise data sources.
Apply production‑readiness practices covering scalability, reliability, security, and performance.
Support CI/CD integration and automated evaluation of AI applications.
Optimize model selection, inference costs, latency, and resource utilization.

Technical Leadership & Collaboration

Provide technical guidance to engineering and QA teams.
Establish reusable frameworks, implementation patterns, and best practices.
Collaborate with architecture, DevOps, QA, and engineering teams.
Participate in design discussions, code reviews, proof‑of‑concept development, and implementation.
Mentor engineers and help teams adopt effective AI development and testing practices.

Required Technical Skills

Agentic AI & Orchestration

LangGraph
LangChain
Amazon Bedrock
Amazon Bedrock AgentCore
AI agents, multi‑agent orchestration, tool calling, workflow management

Generative AI & RAG

Large Language Models (LLMs)
Retrieval‑Augmented Generation (RAG)
Embeddings and vector search
Prompt engineering and context management
LLM application development

LLM Evaluation & Testing

DeepEval
LLM‑as‑a‑Judge techniques
Evaluation metrics and benchmarking
Automated evaluation and regression testing
Assessment of hallucination, faithfulness, relevance, and response quality

AI Observability & Monitoring

OpenTelemetry
LangSmith
Langfuse
LLM tracing, debugging, and performance monitoring

Production Engineering

Deploying and operating production‑grade GenAI applications
API integration and backend engineering
CI/CD and automated testing integration
Scalability, reliability, security, and performance optimization

Preferred Qualifications

8+ years of software engineering or IT experience.
Hands‑on experience developing and deploying GenAI or Agentic AI applications.
Experience with enterprise‑scale AI implementations.
Familiarity with AWS cloud services and enterprise AI security practices.
Experience establishing AI quality standards and evaluation methodologies.
Strong problem‑solving, communication, and technical collaboration skills.

What Success Looks Like

Development and deployment of production‑grade agentic AI applications.
Reliable orchestration and execution of AI agents and workflows.
Effective LLM evaluation and automated AI testing frameworks.
Improved AI application accuracy, reliability, observability, and performance.
Reusable engineering practices for building and maintaining enterprise GenAI solutions.

Important: This is a hands‑on engineering position. Candidates with practical experience building and evaluating AI applications are preferred over those whose experience is limited to AI strategy, architecture documentation, conventional QA automation, or using AI coding assistants.

Flexible work from home options available.

Apply About employer

More about this job

Skills

LangChain
Workflow
Debugging
AWS
CI/CD
Test automation
Problem solving
Communication skills
Collaboration

Report this job