Senior Software Engineer, Apple Services Engineering

Summary The Apple Services Engineering (ASE) team builds the container and virtualization runtime technology that powers Apple's next-generation compute platform. We're a fast-moving, highly skilled team of systems engineers designing and developing the virtualization-based runtimes that run Virtual Machines and Linux Containers across our data center, with privacy, security, and performance as first-class requirements.

We're starting a new team focused on one question: how do we patch the node software across our datacenter fleet faster, and more often, with minimal impact to the workloads running on it? We're looking for a domain expert to decide the approach and then build and lead the team that delivers it.

Description Today, updating a node's software means choosing between speed and workload impact. This team's job is to remove that tradeoff, so we can patch the fleet faster and more often with minimal impact on users. We have not chosen an approach. Live migration, snapshot and restore, kexec-based host updates, live patching, and in-place upgrade of the runtime and VMM are all options, and a different approach may fit better than any of them. We are looking for an expert to decide and guide: which approach, or combination, gets us there, what it costs in workload impact and operational risk, and how we measure the result.

You will:

• Define the fleet patching strategy: evaluate the candidate approaches (including ones we haven't listed), choose, and make the tradeoffs explicit, grounded in a model of how each one fails.
• Set the metrics that matter, such as guest-visible downtime, total migration time, network amplification, abortability, and time to patch the fleet.
• Design and drive the first implementation, building on Cloud Hypervisor and our Linux microVM runtimes on x86 and arm64.
• Hire and grow the team. Depending on your background and interests this is a technical-lead role or a people-management role, and we will decide that together.
• Work with the SRE, Compute, and security organizations on rollout, safety, and fleet operations.

Minimum Qualifications Bachelor's Degree in Computer Science, or equivalent related experience.

At least 8+ years of software design and development experience in infrastructure engineering (compute, storage and network virtualization).

Deep, demonstrated experience in one of the following, and working fluency in the other: Operating live migration, live patching, or kexec-style host updates across a production fleet, including the failures: non-convergence, aborts and rollback, and fleet-wide rollout safety. Building live migration, snapshot/restore, or device-state serialization inside a hypervisor or VMM (for example QEMU/KVM, Cloud Hypervisor, Firecracker, Xen, or Hyper-V).

Advanced programming language proficiency in C, C++, Rust, Swift or Go, with a track record of designing and developing high performance, secured, scalable and reliable systems.

Track record of owning technical design end-to-end, from design docs and architecture tradeoffs through implementation and review.

Experience setting technical direction for a group of engineers, with or without formal management responsibility.

Preferred Qualifications Experience with Linux kernel and system programming (KVM, userfaultfd, VFIO, kexec, kernel live patching). Understanding of dirty-page tracking, memory and device state handling, and time and CPU-model compatibility across hosts. Experience with live migration of devices that carry hardware state (VFIO, vDPA, SR-IOV). Experience designing benchmarks and load tests to characterize downtime and performance impact during an upgrade. Experience contributing to or maintaining open source virtualization projects (Cloud Hypervisor, rust-vmm, QEMU, KVM). Experience with Kubernetes node lifecycle, draining, and fleet rollout orchestration. Experience building a new team from zero, including hiring.

Pay & Benefits At Apple, base pay is one part of our total compensation package and is determined within a range. This provides the opportunity to progress as you grow and develop within a role. The base pay range for this role is between $205,400 and $374,300, and your base pay will depend on your skills, qualifications, experience, and location.

Apple employees also have the opportunity to become an Apple shareholder through participation in Apple’s discretionary employee stock programs. Apple employees are eligible for discretionary restricted stock unit awards, and can purchase Apple stock at a discount if voluntarily participating in Apple’s Employee Stock Purchase Plan. You’ll also receive benefits including: Comprehensive medical and dental coverage, retirement benefits, a range of discounted products and free services, and for formal education related to advancing your career at Apple, reimbursement for certain educational expenses - including tuition. Additionally, this role might be eligible for discretionary bonuses or commission payments as well as relocation. Learn more about Apple Benefits

Note: Apple benefit, compensation and employee stock programs are subject to eligibility requirements and other terms of the applicable plan or program.

EEO Statement Apple is an equal opportunity employer that is committed to inclusion and diversity. We seek to promote equal opportunity for all applicants without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, Veteran status, or other legally protected characteristics. Learn more about your EEO rights as an applicant

Accessibility At Apple, we believe accessibility is a fundamental human right. You’ll find that idea reflected in everything here - in our culture, our benefits and our digital tools. By welcoming as many perspectives as possible, we help you build a career where you feel like you belong.

Learn about accessibility in Apple’s workplace

Learn about reasonable accommodations for job applicants

Application Deadline Apple accepts applications to this posting on an ongoing basis.