Senior / Staff AI Inference Infrastructure Software Engineer

About the Role

We're looking for a senior engineer to own and advance the core of our AI

inference infrastructure — the systems that serve large models at scale. (We

focus exclusively on inference; we don't do training.) You'll set technical

direction in one or more of our focus areas (inference serving, ML compilers, GPU

kernels, communication libraries), ship high-impact performance work, and raise the

bar for the whole team.

This is a deep technical role with ownership and influence. We expect you to go from

a research idea down to PTX, and to make architectural decisions that move

cluster-level inference cost and performance.

What You'll Do

Own a critical part of the stack end-to-end and drive its roadmap:

Inference / serving: continuous batching, paged/prefix KV cache, speculative

decoding, prefill/decode disaggregation, quantization (FP8/INT8/INT4).

Compilers: MLIR / TVM / XLA / TorchInductor passes, fusion, layout selection,

autotuning, codegen for GPUs/accelerators.

Kernels: CUDA / CUTLASS / Triton, Tensor Cores (WMMA/WGMMA), FlashAttention-

class kernels, Hopper/Blackwell features (TMA, warp specialization).

Communication: NCCL internals, custom all-reduce/all-to-all, RDMA/InfiniBand,

NVLink/NVSwitch topology, comm/compute overlap, tensor / pipeline / expert

parallelism for inference.

Deliver measurable wins in latency, throughput, cost, and reliability at scale.

Set engineering standards: perf reviews, benchmarking, regression gates.

Mentor engineers and provide technical leadership across teams.

Represent the team in design reviews and (optionally) externally via talks/writing.

Minimum Qualifications

7+ years building high-performance systems software (or equivalent depth).

Expert-level C++ and CUDA; strong systems fundamentals.

Deep expertise in at least one focus area (serving, compilers, kernels, or

communication) and working knowledge of the others.

Track record of shipping performance-critical systems to production and owning

them operationally.

Experience profiling and optimizing GPU workloads with measurable, significant

impact.

Preferred Qualifications

Experience with high-QPS, low-latency LLM inference serving at scale.

Contributions to major ML systems / open-source projects (vLLM, TensorRT-LLM,

SGLang, Triton, CUTLASS, NCCL, PyTorch, MLIR/TVM/XLA).

Experience with low-precision (FP8/INT4) and modern GPU architectures

(Hopper/Blackwell).

Prior technical leadership / mentoring of engineers.

M.S. / Ph.D. in CS/CE or related, or equivalent industry depth.

Nice to Have

Publications or patents in ML systems / HPC.

Conference talks or technical writing.

Experience bringing up new hardware or custom accelerators.

What We Offer

Ownership of frontier-scale AI infrastructure with direct production impact.

A strong, senior-heavy team and real technical autonomy.

Highly competitive compensation, significant equity, and benefits.