Jobs in the United States

25+ jobs

Save this search

Added just now
Infrastructure Engineer I
Emergent Holdings, Inc.
Minneapolis, MN
The Infrastructure Engineer I role is responsible for the operations of secure and highly available computing platforms, servers, and networks. This role is responsible for installing,…
Added just now
Sr Clinical Specialist, Cranial & Spinal Technologies - Minneapolis, MN
Medtronic
Minneapolis, MN
Our Cranial & Spinal Technologies (CST ) operating unit advances surgical care for spine and cranial conditions through an integrated ecosystem of implants, navigation, robotics, imaging,…
Added just now
SRE- Core Infrastructure Engineer - Professional Level
ManpowerGroup
Minneapolis, MN
SRE/Core Infrastructure Engineer – Professional
Added just now
HR Digital Lead - Learning Digital Solutions
Medtronic
Minneapolis, MN
Experience with vendor governance, service management, and technology support operating models.
Added just now
Toolmaker
Integer Holdings Corporation
Minneapolis, MN
…equipment requirements, and technical advice from supervisor or…
Added just now
Principal Engineer, AI Ecosystem
Google
Sunnyvale, CA · Kirkland, WA · Seattle, WA
Experience operating at a company-wide or multi-organization level, driving technical direction that influences foundational systems.
Added just now
Optical Manufacturing Engineer IV (E4)
Applied Materials
Santa Clara, CA · Santa Clara
As a Manufacturing Engineer, you are responsible for developing and maintaining methods, operating sequences, and processes in the fabrication of parts, components, subassemblies, and final…
Added 2 mins ago
Lead Platform Engineering Architect
GD Information Technology, Inc.
United States
The Lead Platform Engineer designs, builds, and manages baselines that are highly available, secure, and scalable (e.g., the PMO-prescribed Kubernetes distribution and Operating System(s))…
Added 3 mins ago
CF
Clinician, Court-Based
Center for Justice Innovation
Brooklyn, NY
- $61,000 to $78,500 USD / year
Brooklyn Justice Initiatives (BJI), the largest operating project of the Center, seeks to re-engineer the experience of criminal court in Brooklyn, New York, by providing judges and…
Added 3 mins ago
CF
Clinician, Supervised Release Program
Center for Justice Innovation
Staten Island, NY
- $61,000 to $78,500 USD / year
Staten Island Justice Center (SIJC) seeks to re-engineer the experience of criminal court in Staten Island, New York, by providing judges and attorneys meaningful alternatives to bail,…
Added 3 mins ago
CF
Clinician, Community-Based
Center for Justice Innovation
Brooklyn, NY
- $61,000 to $78,500 USD / year
Brooklyn Justice Initiatives (BJI), the largest operating project of the Center, seeks to re-engineer the experience of criminal court in Brooklyn, New York, by providing judges and…
Added 3 mins ago
Sr Supplier Engineer
Intuitive
Sunnyvale, CA
…delivery of purchased production components from suppliers operating under robust quality systems. Each supplier engineer is responsible for ensuring suppliers meet ISI requirements for…
Added 4 mins ago
Cloud Systems Engineer
Dark Wolf Solutions
Ogden, UT
Dark Wolf is seeking a Cloud Systems Operation Engineer to join our team and play a crucial role in maintaining, securing, and optimizing our cloud infrastructure and deployment pipelines.…
Added 4 mins ago
Field Service Technician II - Erie, PA
NCR Atleos
United States
…high-quality service that keeps essential self-service technology operating around the…
Added 5 mins ago
Software Engineering Manager, Infrastructure
Google
Los Angeles, CA
Like Google's own ambitions, the work of a Software Engineer goes beyond just Search. Software Engineering Managers have not only the technical expertise to take on and provide technical…
Added 5 mins ago
Staff Software Engineer, Embedded Systems/Firmware
Google
Sunnyvale, CA
5 years of experience working with embedded operating systems.
Added 5 mins ago
Forward Deployed Engineer, State and Local Government, Google Public Sector
Google
New York, NY
…production-grade, secure AI solutions across Federal and SLED environments. Operating with a high-agency startup mindset, our engineers don’t just advise; they actively code, debug, and…
Added 5 mins ago
Staff Software Engineer, Linux Kernel
Google
Sunnyvale, CA
Experience with computer architecture, operating system and Kernel programming.
Added 7 mins ago
I
Senior Collections Data Engineer (US-Remote)
iCOUNTER
Remote, United States
- $140,000 to $180,000 USD / year
Our AI-native Counter Threat Operating System (CTOS™) combines threat intelligence, risk determination, and operational response to help organizations identify and act on threats involving…
Added 7 mins ago
BV
Cloud Networking Engineer
Bright Vision Technologies
Remote, United States
- $100,000 to $150,000 USD / year
We are looking for a Cloud Networking Engineer to design, deploy, and operate service mesh platforms - primarily Istio and Linkerd - that provide secure, observable, and reliable…
Added 7 mins ago
Container Platform Engineer
Bright Vision Technologies
Remote, United States
As we continue to grow, we’re looking for a skilled Container Platform Engineer to join our dynamic team and contribute to our mission of transforming business processes through technology.
Added 10 mins ago
VI
AI/ML Engineer Team Lead
Vizient, Inc.
Centennial, CO
- $31,533 USD / year
Evaluate architectural tradeoffs across performance, maintainability, extensibility, security, reliability, supportability, and operating cost.
Added 11 mins ago
Senior Support Engineer
CarGurus
Boston, MA
The Senior Support Engineer is the technical specialist inside Customer Operations & Support who closes the gap between "a dealer reports a problem" and "Engineering has a clean ticket to…
Added 11 mins ago
Senior Research & Development Engineer
Synthes USA Products, LLC Legal Entity
West Chester, PA
We are searching for the best talent for a Senior Research and Development Engineer, to join our Orthopaedics team. This is a hybrid role located in West Chester, PA.
Added 12 mins ago
BV
Data Center Virtualization Engineer
Bright Vision Technologies
Remote, United States
- $100,000 to $150,000 USD / year
We are looking for a VMware Platform Engineer to design, deploy, and operate VMware-based virtualization, software-defined networking, and software-defined storage platforms across…

Closed listing

Zenlayer, Inc

Staff Software Engineer, LLM Inference Platform

Remote, United States

This opening has closed.

The original listing is kept below for reference.

Find open jobs More jobs with this employer

About employer

Job details

Posting language: English
Posted: Oct 29, 2026

Similar jobs hiring now

Search current openings

Original job description

Archived listing. The details below describe a past opening.

About Zenlayer

Zenlayer is the first massively distributed edge cloud provider, operating 290+ Points of Presence across 70+ countries. With accolades including PTC “Best Networking Innovation”, “Outstanding Cloud, Data Center, or Interconnection Company”, and Inc. 5000, we enable businesses to deploy and run applications at the edge, thus reducing network latency to the end user. Zenlayer focuses on improving connections worldwide, all while nurturing a unique company culture that values service, growth, and trust. As a “Zenplayer”, you’ll be joining a community of success-driven team players that share the dream of powering a better-connected world.

About the Role:

As the technical lead for inference performance, you will own end-to-end responsibility for the platform's Token Performance. For models ranging from hundreds of billions to trillions of parameters (including MoE), you will continuously reduce cost per million tokens and increase per-GPU Goodput while meeting TTFT/TPOT SLOs, and prove the gain from every optimization with reproducible data.

What You Will Work On

End-to-End Performance Attribution and Benchmarking

Establish a unified Token Performance metrics framework (TTFT, TPOT, Goodput, GPU utilization, per-token cost), and build load-testing benchmarks that reflect real production traffic (high concurrency, long context, multi-turn Agent workloads)
Perform bottleneck attribution for models such as DeepSeek, Kimi, GLM, Qwen, and MiniMax across compute, memory bandwidth, communication, scheduling, and queuing, and let the data determine which layer is the dominant constraint

Kernel-Level Inference Engine Optimization

Dive deep into the source code of vLLM / SGLang / TensorRT-LLM, customizing and contributing upstream on critical paths such as the scheduler, Continuous Batching, Chunked Prefill, CUDA Graph, and KV block management
Clearly define the responsibility boundary between the inference engine and orchestration layers such as Dynamo / llm-d, and reduce tail latency through request routing and load balancing

PD Disaggregation and KV Cache Transfer

Lead the core design of Prefill-Decode disaggregation: the KV transfer path (NIXL/RDMA/UCX), transfer-compute overlap, and phase-aware autoscaling
Define the failure domains and recovery semantics of cross-instance KV transfer, and answer the key question: under which workloads does PD disaggregation deliver gains, and under which does it make things slower

MoE Parallelism and Wide-EP

Design hybrid TP/PP/DP/EP parallelism strategies and quantify the trade-offs between EP and TP
Tackle expert load imbalance and All-to-All communication overhead, and drive Wide-EP into production

Kernels and Quantization

Write and tune critical CUDA (and ROCm/HIP) operators: memory access optimization, kernel fusion, and Attention / MoE / communication-related kernels
Build a closed-loop "quantization – accuracy – throughput" evaluation: the real-world gains and accuracy regressions of FP8/FP4/INT4/AWQ/GPTQ across different engines and hardware

Requirements

First-principles understanding of Transformer inference: the compute vs. memory-bound nature of Prefill and Decode, the KV Cache memory model, the differences among MHA/GQA/MLA, and how MoE routing constrains parallelism
Source-level, in-depth experience with at least one mainstream inference engine, with real cases of identifying and fixing bottlenecks in scheduling, kernel selection, or KV management
CUDA kernel tuning skills (Nsight profiling, memory optimization, fusion, CUDA Graph) with hands-on practical experience
Bachelor’s degree in Computer Science, Electrical/Computer Engineering, or related field (HPC systems, ML Systems).
Minimum 6 years in systems/Performance Engineering, with at least 2-3 years directly on LLM or GPU performance work

Preferred, but not required:

Experience with Merged PRs in vLLM
Experience operating at cluster sale (hundreds to thousands of GPUs)

About employer

Report this job