Senior Staff Engineer - AI ADC

Required Skills & Experience

Advanced degree in Computer Science, Networking, or related field (MS/PhD preferred) Deep knowledge of L4–L7 protocols (TCP/TLS/HTTP/2/3, QUIC), load balancing algorithms and GSLB strategies (DNS‑based, HTTP redirect, anycast) Strong programming skills in C/C++ (data plane), Go/Python (control/ML), and performance profiling (perf, eBPF, flamegraphs, VTune/nvprof)

Hands‑on experience with AI/ML systems, including:

-Model inference pipelines, LLM API integrations -Token‑level behavior and performance characteristics -Understanding of AI workload routing challenges (latency, caching, batching, multi‑model orchestration)

Job Description

A global technology client of Insight Global is seeking a Senior Staff Engineer - AI ADC to join their team. This company specializes in high-performance application delivery, network security, and AI-ready infrastructure solutions for enterprises, service providers, and government organizations.

This role combines deep domain expertise in Application Delivery Controllers (ADC), Server Load Balancing (SLB), DNS/GSLB, with practical experience in AI/LLM systems, to build intelligent routing, inference‑aware load balancing, and Layer‑8 contextual decision engines for AI workloads. The ideal candidate is a senior staff engineer who can drive architecture, design, prototyping, and execution - working closely with product management, platform teams, and AI strategy groups.

Key Responsibilities Include:

-Lead architecture and development of AI Gateway components for intelligent routing of LLM/AI application traffic. -Design and implement LLM‑aware load balancing, incorporating semantic, token, latency, and model‑level insights into traffic decisions.

-Develop enhancements for ADC and GSLB platforms to support AI workloads, including:

Token‑aware rate limiting Inference latency‑based routing AI model endpoint discovery and health checks Integration with vector databases, model registries, and ML observability systems

-Contribute to advanced features including Layer‑8 context‑aware routing, adaptive traffic shaping, and AI‑driven anomaly detection. -Drive system‑wide architecture: data plane, control plane, configuration, and distributed state management.

This role is permanent and will be worked hybrid on-site in San Jose, CA. We are looking to pay between $180,000-$195,000 base salary. Apply today!