AI Engineer Lead
- San Jose, CA, United States
Applications open in a new tab.
Job details
- Experience
- 8+ years of experience in Software Engineering, ML Engineering, MLOps, or DevOps, including 2+ years in a technical leadership role.
- Posting language
- English
- Posted
Original job description
- Help design, build and continuously improve the clients online platform.
- Research, suggest and implement new technology solutions following best practices/standards.
- Take responsibility for the resiliency and availability of different products.
- Be a productive member of the team.
Requirements
- 8+ years of experience in Software Engineering, ML Engineering, MLOps, or DevOps, including 2+ years in a technical leadership role.
- Experienced ML/MLOps Technical Lead with strong expertise in machine learning operations, GenAI, Agentic AI, and production ML infrastructure.
- Lead globally distributed engineering teams, drive end-to-end ML lifecycle management, and ensure scalable, secure, and reliable AI model deployment, evaluation, and monitoring.
- Define technical strategy and roadmap for ML sustenance across multiple services.
- Act as the primary onsite technical contact and lead globally distributed engineering teams.
- Drive architecture reviews, define Service-Level Objectives (SLOs), and coordinate with client teams for AI agent onboarding.
- Standardize model evaluation processes and mentor engineering teams.
- Design, develop, and maintain automated Continuous Training (CT), data ingestion, and feature engineering pipelines.
- Work with MLOps frameworks and tools such as Kubeflow, MLflow, and Apache Airflow.
- Integrate datasets from distributed SQL systems and messaging platforms such as Apache Kafka.
- Implement model, prompt, and evaluation configuration versioning, experiment tracking, and hyperparameter tuning.
- Build isolated agent execution environments using Kubernetes and Docker.
- Develop evaluation pipelines, synthetic datasets, golden datasets, and simulated execution traces.
- Establish quality dashboards and automated monitoring using metrics such as Accuracy, AUC, ROUGE, and Trajectory Exact Match.
- Implement alerting mechanisms to detect model drift, hallucinations, and quality degradation.
- Evaluate and monitor automated assessment frameworks, including LLM-as-a-Judge.
- Build and maintain CI/CD pipelines for automated ML model deployment and rollback.
- Implement monitoring across metrics, logs, and traces to ensure production reliability and low-latency model serving.
- Handle production incidents, troubleshooting, traffic rollouts, and root-cause analysis.
- Optimize system performance, resolve capacity bottlenecks, and improve MTTR.
- Implement data isolation mechanisms for sensitive and multi-tenant environments.
- Enforce IAM, RBAC, encryption, and Zero Trust security principles.
- Ensure compliance with data retention, access control, and deletion policies.
- Develop AI-driven operational automation, including monitoring agents, automated incident triage, and documentation synchronization.
- Optimize infrastructure, compute, and token costs across model serving and evaluation workloads.
- Strong expertise in GCP MLOps services, particularly Vertex AI and Kubeflow, or equivalent tools such as MLflow and Apache Airflow.
- Proficiency in Python or Go, distributed systems, and large-scale data processing.
- Hands-on experience with Kubernetes, Docker, containerized deployments, and CI/CD tools such as Jenkins or GitLab CI.
- Strong understanding of end-to-end ML pipelines covering data ingestion, model training, evaluation, deployment, and continuous monitoring.
- Experience in production operations, SLO definition, monitoring, alerting, incident management, and root-cause analysis.
- Proven experience leading distributed teams and managing technical and business stakeholders.
- Knowledge of LLMs and Agentic AI frameworks such as LangChain, AutoGen, LlamaIndex, or Google Agent Development Kit (ADK).
- Experience with ML and GenAI evaluation metrics, including BLEURT, ROUGE, Precision, Recall, AUC, and Trajectory Exact Match.
- Experience handling sensitive data such as PII/PHI with strong encryption and access controls.
- Knowledge of model monitoring, data/concept drift detection, A/B testing, and quality dashboards using Grafana or Datadog.
- Experience with production support, ML sustenance, SRE, service transition, and knowledge-transfer programs.
Benefits
- A challenging, innovating environment.
- Opportunities for learning where needed.
More about this job
Skills
- Research
- ML
- DevOps
- Machine learning
- Onboarding
- Kubeflow
- Mlflow
- Apache Airflow
- SQL
- Apache Kafka
- Kubernetes
- Docker
- CI/CD
- Troubleshooting
- Root cause analysis
- Identity and access management
- Regulatory compliance
- Documentation
- Google Cloud
- Python
- Go
- Process data
- Jenkins
- GitLab
- LangChain
- Grafana
- Datadog