AI Engineer

As an AI Engineer at Vytalize Health, you will design, build, and maintain agentic systems and LLM-powered applications that automate complex healthcare workflows and accelerate our ability to deliver data-driven clinical solutions. Working at the intersection of applied AI and healthcare, you will build agents that orchestrate data retrieval, model inference, clinical logic, and tool use to solve problems that traditionally require manual effort or specialized expertise.

You will work cross-functionally with data engineering, platform, product, and clinical teams to identify high-impact opportunities for AI automation—from data source onboarding to clinical decision support to evidence synthesis. Your focus will be on building production-grade agentic systems with rigorous validation, clear confidence scoring, and human-in-the-loop oversight to ensure reliability in a regulated healthcare environment.

You will establish patterns, best practices, and tooling that allow the organization to scale AI-driven automation across multiple domains. You will measure agent performance and impact—tracking accuracy, hallucination rates, and real-world clinical outcomes.

Primary Responsibilities

• Design, build, and maintain agentic systems and LLM-powered applications that automate healthcare workflows, data pipelines, and clinical decision support — from conception through production deployment

• Build and orchestrate agents using LLM APIs (OpenAI, Anthropic, etc.) and agentic frameworks (LangChain, LangGraph, CrewAI, or custom orchestration) to solve complex, multi-step healthcare problems

• Develop prompt libraries, agent instructions, and reusable "skills" that improve agent accuracy, consistency, and reliability across different use cases and data domains

• Build validation and confidence-scoring layers that flag low-confidence agent decisions for human review before production deployment; establish guardrails and review workflows for agent-authored code and outputs

• Own end-to-end delivery of AI-automated systems — from problem scoping and requirements gathering through agent development, testing, and validated production deployment

• Implement rigorous evaluation and QA frameworks for agentic systems — including golden datasets, test cases, output validation, hallucination detection, and regression testing

• Establish and maintain evaluation metrics for agent performance, reliability, and clinical appropriateness; measure agent accuracy, hallucination rates, clinical validity, and real-world impact

Required Qualifications

• 3+ years of professional experience in data engineering, backend engineering, machine learning, or a related field

• 1+ years of hands-on experience building with LLM APIs and agentic orchestration frameworks — not just using AI coding assistants, but architecting agentic systems

• Strong Python and SQL proficiency