Machine Learning Cloud Infrastructure Engineer

As a Machine Learning Cloud Infrastructure Engineer, you will build and operate the infrastructure HavocAI teams use to train, evaluate, deploy, and monitor machine learning models safely and reliably.

You will develop the pipelines, services, platforms, and integrations connecting data lakes, telemetry stores, simulation environments, training workloads, cloud compute, and deployed models. Your work will enable autonomy, data, and software engineers to move efficiently from raw field and simulation data to reproducible datasets, scalable training, rigorous evaluation, and production deployment.

This role is ideal for a strong software, infrastructure, or data engineer with hands-on ML experience who enjoys building production systems at the intersection of data, models, compute, and autonomy. You should be comfortable operating in a fast-paced environment, solving ambiguous infrastructure problems, and building systems that must remain scalable, observable, secure, and reliable.

ML Infrastructure & Pipelines

• Build pipelines that transform raw multi-modal data—including telemetry, imagery, video, sensor, and simulation data—into curated, versioned training datasets.

• Develop reproducible training and evaluation workflows that scale across cloud compute and GPU resources.

• Build and maintain model deployment infrastructure for packaging, serving, inference, versioning, and rollback.

• Implement experiment tracking, dataset lineage, model versioning, and other capabilities required for reproducible ML development.

• Own data schema versioning and migration across pipelines, data lakes, and services as datasets and models evolve.

Cloud Platform & Infrastructure

• Design, build, and operate scalable AWS infrastructure using Infrastructure as Code.

• Build and maintain Kubernetes/EKS workloads and containerized environments for training, batch processing, evaluation, and model serving.

• Develop self-service tooling and paved paths for compute scheduling, storage, data access, training, and deployment.

• Improve utilization, scalability, and cost efficiency across cloud and accelerator infrastructure.

• Build infrastructure that enables engineering teams to launch workloads safely without unnecessary operational overhead.

Reliability, Evaluation & Observability

• Build evaluation frameworks and regression testing for model quality, dataset integrity, and pipeline correctness.

• Establish quality and reliability signals that help determine whether models are ready for production use.

• Develop monitoring, logging, tracing, and observability across training jobs, data pipelines, and deployed models.

• Diagnose and resolve performance, scaling, reliability, and infrastructure bottlenecks.

• Maintain high standards for automation, testing, documentation, and operational readiness.

Cross-Functional Engineering

• Partner with Autonomy, Software, Data, Simulation, and Security teams to build ML infrastructure spanning edge data capture through cloud training and model deployment.

• Contribute to CI/CD and release processes for models, datasets, and ML pipelines.

• Translate engineering requirements into scalable platform capabilities that can support multiple teams and use cases.

• Incorporate feedback from engineers and users to continuously improve ML development workflows.

Security & Data Management

• Implement secure infrastructure practices, including IAM least privilege, secrets management, access controls, and secure handling of sensitive and defense-related data.

• Build data and ML workflows with reproducibility, traceability, and appropriate controls from the start.

• Partner with security and infrastructure teams to ensure ML systems meet applicable operational and compliance requirements.