OpeningsFirst
How it worksPricingLog in
OpeningsFirst › Jobs in the United States

Inference infrastructure Engineer

Acceler8 · acceler8talent.com
Middlesex County
Apply on Acceler8’s site

Straight from the company’s own careers site.

The job in short

AI inference infrastructure engineering.

LLM

Details

Found
6 d ago · posted 2026-10-28
Location
Middlesex County, Massachusetts

Full description

I'm hiring a Member of Technical Staff, ML Systems to join an early-stage AI infrastructure company building a new memory layer for large-scale inference. The team is developing custom silicon and systems software that give GPUs and other AI accelerators access to a much larger high-bandwidth memory tier, with a particular focus on long-context inference, KV-cache management and reducing the cost of keeping large amounts of model state close to compute. This is a low-level ML systems role working across inference engines, GPU performance and hardware/software co-design rather than application-layer AI. → Build and optimise LLM inference systems around distributed and expandable memory → Work across KV cache, data movement, GPU kernels and serving performance → Help shape the underlying hardware architecture using real workload behaviour The team is particularly interested in engineers with experience around vLLM, SGLang, TensorRT-LLM, LMCache, CUDA or similar inference infrastructure. If this jobs sounds like something you would be interested in, dont hesitate to reach out!

DevOps / SRE / Platform Engineer jobs in the United States · All Acceler8 jobs

Get jobs like this every morning

Jobs in India · Jobs in the US · Who’s hiring in India · Who’s hiring in the US · How OpeningsFirst works · Free jobs widget
OpeningsFirst · Use with AI · Terms · Privacy · Refunds · Contact Showing prices for outside India or India