Job Description:
Senior ML Engineer
Location: Ann Arbor, Michigan (Remote)
Role Overview
This position is for a Senior Machine Learning Engineer to join a team focused on developing end-to-end Vision-Language-Action (VLA) models for autonomous vehicles. The team is responsible for converting petabytes of multi-modal fleet data into high-quality, semantically rich training datasets. This role will own the dataset layer that these advanced models learn from, playing a part in the data flywheel that includes mining failure cases, auto-labeling at scale, and validating data quality.
Key Responsibilities
- Design, implement, test, and deploy cloud-based pipelines to convert logged multi-sensor data into VLM/VLA training datasets.
- Develop open-vocabulary detection, dense captioning, and semantic enrichment using foundation models to scale annotation.
- Produce language-grounded reasoning and chain-of-causation style annotations to support VLA training.
- Surface rare, difficult, and high-uncertainty scenarios to build curated datasets that improve downstream model metrics.
- Define dataset schemas, quality metrics, and validation processes, tracking auto-labeling quality against model requirements.
- Partner with VLM/VLA model developers to define dataset specifications and operationalize a continuous delivery loop.
- Build distributed, reproducible pipelines using columnar data formats and distributed compute.
- Serve as a project lead, mentor engineers, conduct design reviews, and set coding and annotation standards.
- Track advances in multimodal models, auto-labeling, and end-to-end autonomous driving to apply relevant research.
Required Qualifications
Education: A Bachelor's Degree in Computer Science, Robotics, Electrical Engineering, or a related technical field with competences typically acquired through 6+ years of experience, or a Master's Degree in a related technical field with competences typically acquired through 3+ years of experience.
Experience: Must be highly skilled and proficient, able to conduct complex work under minimal supervision. Expected to drive alignment across team interfaces, own technical solutions, and mentor other engineers.
Technical Skills: A combination of the following skills is required:
- Computer Vision & Deep Learning: Model training and expertise in at least two of the following: 2D/3D Object Detection, Tracking, Sensor Fusion, Semantic Segmentation, BEV, or Depth Estimation.
- Multimodal / VLM Experience: Hands-on work with vision-language models, open-vocabulary recognition, dense captioning, or semantic search applied to perception data.
- Model Data Curation: Building targeted datasets and large-scale Parquet data processing (e.g., Databricks, Daft, Pandas).
- Distributed ML & Data Frameworks: Experience with PyTorch, Lightning, Ray, Spark, or equivalent.
- Scaled MLOps & Tooling: Knowledge of experiment tracking, model registry (e.g., MLflow, Weights & Biases), and ML metrics.
- Development Tools: Strong Python software development skills, experience with VDI and cloud-based environments, CI systems (e.g., GitHub Actions), and Docker.
Preferred Qualifications
Familiarity with VLM/VLA or end-to-end driving models and reasoning-trace datasets.
Experience with auto-labeling foundation models for annotation and verification.
Knowledge of high-throughput model serving frameworks like vLLM or SGLang.
Experience with semantic inference, retrieval, and vector databases such as LanceDB.
Familiarity with AV data standards (e.g., Pegasus layers) and robotics formats