AI Inference Infrastructure Software Engineer

About the Role

We're a team focused exclusively on AI inference — building the systems that

serve large models fast, efficiently, and reliably at scale. (We don't do training;

our entire stack is dedicated to making inference better.) As a Junior AI Inference

Infrastructure Engineer, you'll work alongside senior engineers on the core software

stack — inference serving engines, ML compilers, GPU kernels, and communication

libraries. This is a hands-on role for someone early in their career who loves

systems programming and wants to go deep on GPU inference performance.

You don't need to be an expert in all of our focus areas — strong fundamentals and

a demonstrated ability to learn fast matter more.

What You'll Do

Implement and optimize components of our ML inference stack

(one or more of: serving engine, compiler, GPU kernels, collective communication).

Write and tune CUDA / Triton kernels; profile with Nsight and chase down

bottlenecks.

Build benchmarks and regression tests to keep performance honest.

Contribute bug fixes and features to internal systems and relevant open-source

projects (e.g., vLLM, Triton, PyTorch).

Read papers and translate them into working, measurable code.

Minimum Qualifications

B.S. or M.S. in Computer Science, Computer Engineering, or related field (or

equivalent practical experience).

Solid programming skills in C++ and Python.

Fundamentals in computer architecture, operating systems, and parallel

computing.

Exposure to GPU programming (CUDA / Triton) through coursework, projects,

internships, or open source.

Comfort with Linux, Git, and a debugger/profiler.

Preferred Qualifications

Internship or project experience in one of our focus areas:

Inference / serving: batching, KV-cache, quantization, latency optimization

Compilers: MLIR / TVM / XLA, torch.compile, codegen, op fusion

Kernels: CUDA/Triton/CUTLASS, Tensor Cores, FlashAttention-style kernels

Communication: NCCL, all-reduce / all-gather, tensor / pipeline parallelism for serving

Open-source contributions to ML systems projects.

Experience profiling and optimizing real GPU workloads (measurable wins).

Familiarity with PyTorch internals.

Nice to Have

Reading PTX / assembly; comfort with low-level performance work.

A personal project (e.g., a from-scratch inference engine or kernel library).

What We Offer

Mentorship from senior/staff infra engineers and a real growth path.

Work on frontier-scale AI systems with real production impact.

Competitive compensation, equity, and benefits.