AI/ML Engineer (SageMaker & AWS Trainium)

Robots & Pencils is an AWS Partner building production AI systems for enterprise clients who need real engineering, not proofs of concept that never ship. We work forward-deployed, embedded directly with client teams, solving the problems that are too new or too specialized for a typical vendor relationship to handle.

What You'll Do

• Train and operate models on Amazon SageMaker with AWS Trainium as the underlying compute
• Write and optimize PyTorch training code with a real understanding of how it compiles and executes on Trainium (NeuronCore architecture, compiler behavior, memory and throughput tradeoffs)
• Diagnose training run issues that show up specifically because of the hardware, not just the model, distinguishing a data or code problem from a compiler or device-level one
• Translate a request for "a Trainium job" into an actual working, cost-aware training pipeline, end to end
• Tune distributed training runs for throughput and cost on SageMaker’s training infrastructure
• Work directly with client and internal engineering teams to scope and deliver real production training workloads, not experiments that stay in a notebook

What You'll Bring

• Strong, hands-on PyTorch experience, ideally including distributed or multi-device training
• Production experience with Amazon SageMaker for training and/or inference
• Comfort working close to the hardware layer: you understand device-specific compilation and can debug issues that are actually about the accelerator, not just the model
• AWS Trainium or Inferentia (Neuron SDK) experience is a strong plus; if you don’t have it yet but have deep PyTorch and a track record of picking up new hardware targets fast, we want to talk to you
• Solid Python fundamentals and comfort operating in a client-facing, production engineering environment