Jobs in the United States

25+ jobs

Group jobs by employerEmail me new jobs Save this search

Added just now
Test Site Engineer
Noblis
White Sands, NM
Noblis is hiring a Test Site Engineer at White Sands, NM. This position provides test engineering support by developing test plans, procedures, and data collection requirements; translating…
Added just now
State Construction Engineer
Arizona Department of Administration
AZ
Under the general direction of the Deputy State Engineer for Operations this position will provide executive-level oversight and technical leadership for the state's construction program.…
Added 3 mins ago
Full-Stack Engineer (Java/React/Mobile) - 3711500
Lighthouse Technology Services
Buffalo, NY
Lighthouse Technology Services is partnering with our client to fill their Full-Stack Engineer (Java/React/Mobile) position! This is a 3-4 month contract with possible extension and will be…
Added 3 mins ago
Software Development Engineer, Amazon Leo Live
Amazon Kuiper Manufacturing Enterprises LLC
Redmond, WA
We are looking for a engineer with experience in supporting mission-critical applications, who are excited about building solutions that enhance automation, stability, and release processes…
Added 3 mins ago
Software Development Engineer - Embedded, Flight Computer Software
Amazon Kuiper Manufacturing Enterprises LLC
Redmond, WA
- 2+ years of non-internship design or architecture (design patterns, reliability and scaling) of new and existing systems experience
Added 3 mins ago
Kernel / Hypervisor Engineer - Firecracker, Firecracker
Amazon Development Center U.S., Inc.
New York, NY
Your main goal will be to create long-term customer value. You will: work to understand our customers' needs and engineer new features to meet them; own and automate operations; contribute…
Added 3 mins ago
Program Manager, Data Center Community
Amazon Data Services, Inc.
Hobart, IN
…cluster wide maintenance coordination. An understanding of these systems is necessary, as the individual will be expected to identify areas for improvement as well as act on data provided…
Added 4 mins ago
FE
Data Engineer II
Federal Express Corporation
Plano, TX
- $7,094.23 to $15,607.31 USD / month
Candidates must live within 50 miles of the campus location
Added 6 mins ago
CM
Senior IAM Engineer
Centene Management Company LLC
Remote, United States
- $87,000 to $161,300 USD / year
…Access Management (IAM) and Multi-Factor Authentication (MFA) systems and solutions. Performs administration, configuration, integration, and maintenance of authentication platforms,…
Added 7 mins ago
Forward Deployed Engineer (FDE)
Gravitee
Denver
…Roche, and Blue Yonder use Gravitee to control and observe systems their businesses depend…
Added 11 mins ago
Private Cloud Engineer III
Tierpoint
Remote, United States
…deep technical expertise, demonstrate ownership of complex systems, and actively mentoring junior team members. Regular interaction with internal teams and clients is expected, with an…
Added 11 mins ago
Database and App System Administrator
Temple University
PA
Temple University Database and Application team is seeking a passionate engineer professional interested in Information Technology and application/database support. This is an opportunity…
Added 13 mins ago
Sr. Equipment Engineer
LG Energy Solution Michigan, Inc.
Holland, MI
…battery solutions for electric vehicles (EVs) and Energy Storage Systems (ESS). With expanding operations across the United States, we are committed to innovation, quality, safety, and…
Added 13 mins ago
A
Golang Software Engineer
Actalent
Raleigh, NC
- $50 to $60 USD / hour
Job Title: Golang Software Engineer (Mid-Expert Platform Engineering)Job DescriptionThis contract role offers an opportunity for a mid-expert level Golang Software Engineer to join a…
Added 14 mins ago
EL
Structural Senior Engineer
ECS Ltd
Chantilly, VA
- $90,000 to $120,000 USD / year
Professional Engineer (P.E.) license required · Valid driver’s license required if driving for or on behalf of an ECS subsidiary
Added 15 mins ago
Sr. Facilities Engineer
LG Energy Solution Arizona
Queen Creek, AZ
The Sr. Engineer, Facilities is responsible for managing and overseeing the utility systems at the battery plant. This includes managing critical infrastructure such as electrical, water,…
Added 16 mins ago
Mechanical Engineering Manager
CAIRE
Saint Paul, MN
MGC Diagnostics’ non-invasive cardiorespiratory systems help clinicians measure heart and lung function and make better care decisions. Mechanical design sits at the core: precision airflow…
Added 16 mins ago
Missile Warning Systems Engineer (TS clearance)
Vantor Services Inc.
El Segundo, CA
Onsite attendance is required at least 2–3 days per week
Added 16 mins ago
Mission Engineer - Space Support to Theater (TS clearance)
Vantor Services Inc.
El Segundo, CA
Vantor is seeking a Mission Engineer to support the U.S. Space Force and our teammates in integrating space capabilities into theater operations. Supporting Space Systems Command (SSC) and…
Added 17 mins ago
Lead Technician, Maintenance
Merck Sharp & Dohme LLC
NC
This is a night shift position 12 hour (6:00pm - 6:00am) rotation (2-2-3) and a day shift position M-F
Added 17 mins ago
Associate Automation Engineer
Amneal
Branchburg Township, NJ
The Automation Engineer's primary role is to work under supervision and coordinating installation, troubleshooting and assist in maintaining automated processes and facility related…
Added 20 mins ago
PC
IT Engineer - Telecom
Pinal County
Florence, AZ
- $34.12 to $47.77 USD / hour
Perform engineering studies involving the various components of information technology systems.
Added 21 mins ago
IG
Secret Kubernetes Engineer
Insight Global
North Charleston, SC
- $100,000 to $150,000 USD / year
- Must have an active Secret level security clearance
Added 21 mins ago
Senior Software QA Engineer, Avionics Platform
Firestorm
United States
role at our San Diego HQ. Expect meaningful hands-on time with benches, flight computers, power systems, radios, and sensors to validate end-to-end flight workflows.
Added 22 mins ago
NA
Data Centre Service Delivery Field Support Engineer (L2)
NTT America, Inc.
NE
- $29.33 to $41.83 USD / hour
The Data Center Service Delivery Field Support Engineer (L2) is a developing engineering role, responsible for providing a professional second-line onsite technical support and field…

Bask Health

Machine Learning Engineer, Evaluation & Reliability

Remote, United States

Apply on apply.workable.com About employer

Applications open in a new tab.

Job details

Experience: 3+ years of professional experience in machine learning engineering, software engineering, applied AI, or a closely related technical field.
Posting language: English
Posted: Nov 12, 2026

Original job description

About Bask Health Bask Health is a software platform that enables entrepreneurs, doctors, and healthcare businesses to launch, manage, and grow direct-to-consumer telehealth businesses. Our platform brings together patient experiences, clinical workflows, pharmacy operations, and commerce tools in one place. We're an entrepreneurial, product-focused engineering team that values ownership, technical excellence, and building software that solves real problems. Engineers work closely with product teams and company leadership, take features from idea to production, and have a direct role in shaping the platform. Engineering at Bask is AI-first and builder-led. We use tools like Cursor and Claude Code throughout development while maintaining high standards for reliability, testing, and software quality. About the Role We're looking for a Machine Learning Engineer, Evaluation & Reliability to help ensure that AI-powered features across Bask Health are accurate, dependable, and ready for real-world use. Bask Health uses software to connect patient experiences, merchant operations, clinical workflows, pharmacy coordination, and commerce capabilities. As AI becomes more integrated into these systems, we need reliable ways to measure model performance, identify failures, and verify that changes improve the user experience without introducing new risks. You'll help develop the evaluation infrastructure, testing workflows, and performance measurements used to assess large language models, retrieval systems, AI assistants, and automated workflows. This is a hands-on engineering role that combines machine learning evaluation, software development, data analysis, and production quality assurance. You'll work with model outputs, evaluation datasets, automated testing tools, and application telemetry to understand how AI systems perform under realistic conditions. Your work may include creating evaluation datasets, implementing automated benchmarks, measuring response quality, testing retrieval accuracy, analyzing model failures, and monitoring production behavior. You'll collaborate with applied AI engineers, machine learning engineers, backend engineers, product managers, and other stakeholders to define what successful AI behavior looks like for different workflows. You'll help translate those expectations into measurable tests and actionable engineering improvements. Because Bask operates in healthcare-related environments, AI evaluation must account for accuracy, privacy, appropriate permissions, and the consequences of unreliable responses. You'll help identify situations where AI systems produce unsupported information, expose restricted data, attempt unauthorized actions, or operate outside approved clinical and operational boundaries. This position emphasizes practical implementation and continuous improvement. You'll contribute to evaluation methodologies and engineering standards while working with senior engineers on broader reliability strategies and more complex technical decisions. We're looking for an engineer who enjoys investigating how AI systems behave, can turn ambiguous quality problems into measurable tests, and understands that reliable AI products require ongoing evaluation - not just successful demonstrations. What You'll Do

Build and maintain evaluation systems for AI-powered applications across Bask Health.
Develop automated testing frameworks for large language models and AI application workflows.
Create and maintain representative datasets for evaluating model behavior.
Implement benchmarks that measure response accuracy, relevance, completeness, and consistency.
Develop evaluation pipelines for prompts, retrieval systems, and model configurations.
Test AI assistants and agents against realistic user scenarios and workflow requirements.
Build automated regression tests to detect performance changes between releases.
Measure hallucination rates, unsupported claims, and other model quality failures.
Evaluate retrieval-augmented generation systems for document relevance and answer grounding.
Develop tests for structured model outputs and schema compliance.
Evaluate tool-calling accuracy and agent action selection.
Test AI workflows for unauthorized actions, incorrect permissions, and unsafe behavior.
Implement evaluation checks for sensitive patient and operational information.
Build dashboards and reports that communicate AI quality metrics to engineering teams.
Analyze evaluation results and identify recurring failure patterns.
Investigate production incidents involving inaccurate or unreliable AI outputs.
Collaborate with applied AI engineers to improve prompts, retrieval logic, and application behavior.
Support model comparisons and controlled experiments.
Evaluate tradeoffs between model quality, latency, reliability, and inference cost.
Develop automated quality checks for CI/CD and AI feature release workflows.
Implement monitoring and alerting for production AI performance.
Build tools for reviewing, labeling, and analyzing model responses.
Help maintain evaluation datasets as products and user behavior evolve.
Document evaluation methodologies, test cases, and known model limitations.
Work with product teams to define measurable acceptance criteria for AI features.
Collaborate with senior engineers on evaluation standards and reliability improvements.
Participate in code reviews, technical discussions, and engineering planning.
Own assigned evaluation and reliability projects from implementation through deployment and ongoing improvement.

Requirements

3+ years of professional experience in machine learning engineering, software engineering, applied AI, or a closely related technical field.
Experience building and maintaining production software or machine learning applications.
Strong programming skills in Python, TypeScript, or comparable languages.
Familiarity with large language models and modern generative AI applications.
Experience testing or evaluating machine learning models or AI-powered software.
Understanding of machine learning evaluation metrics and experimental methodologies.
Experience developing automated tests and validation pipelines.
Familiarity with prompt engineering and model output analysis.
Ability to analyze structured and unstructured datasets.
Experience writing scripts or services for data processing and evaluation.
Understanding of model accuracy, precision, recall, and other performance measures where applicable.
Familiarity with API integrations and backend application development.
Experience working with relational databases or analytical data systems.
Ability to identify patterns in model failures and translate findings into engineering improvements.
Understanding of regression testing and software quality assurance.
Familiarity with logging, monitoring, and production troubleshooting.
Experience using Git and collaborative software development workflows.
Understanding of the limitations and failure modes of large language models.
Ability to design representative test cases and evaluation scenarios.
Familiarity with cloud-based development and deployment environments.
Experience writing maintainable, well-tested production code.
Strong analytical thinking and attention to detail.
Ability to communicate technical evaluation results clearly.
Experience collaborating with engineers, product managers, and cross-functional teams.
Ability to independently own assigned technical projects.

Nice to Have

Experience building evaluation frameworks for large language models.
Familiarity with OpenAI, Anthropic, or comparable model APIs.
Experience with tools such as LangSmith, DeepEval, Ragas, or comparable AI evaluation frameworks.
Familiarity with LLM-as-a-judge evaluation techniques.
Experience creating human-labeled evaluation datasets.
Understanding of inter-rater agreement and evaluation consistency.
Experience evaluating retrieval-augmented generation systems.
Familiarity with embeddings, semantic search, and vector databases.
Experience evaluating AI agents and tool-calling workflows.
Familiarity with prompt injection testing and AI security evaluations.
Experience implementing automated regression testing for AI applications.
Familiarity with experiment tracking and model comparison tools.
Experience building dashboards for machine learning performance monitoring.
Understanding of statistical analysis and experiment design.
Experience working with Python data analysis libraries.
Familiarity with TypeScript and Node.js application development.
Experience working with AWS or serverless infrastructure.
Experience evaluating AI systems used in healthcare or other sensitive environments.
Understanding of HIPAA and healthcare data privacy considerations.
Experience using AI-assisted development tools such as Cursor or Claude Code.

Apply on apply.workable.com About employer

Report this job

Work requirements

Jobs in the United States