Senior AI Product Engineer

We're looking for a Senior AI Product Engineer to own the continuous improvement of our customer-facing AI products, making them more accurate, reliable, cost-efficient, and genuinely useful to the people who rely on them every day. This isn't a prompt-writing job, it's about building the structured loop of evaluation, experimentation, and analytics that turns AI from a feature into a measurable product advantage. You'll work at the intersection of Product, Growth, and Engineering, with real influence over how AI behavior gets measured, improved, and shipped.

What You’ll Do

• Own the end-to-end quality of our AI-powered product experiences, improving response usefulness, consistency, and reliability in ways users actually notice.
• Build evaluation frameworks and repeatable test sets so we can measure AI quality objectively, not anecdotally.
• Design and run experiments on prompts, context strategies, and model settings, connecting every change to activation, conversion, retention, or satisfaction metrics.
• Monitor and reduce AI-related costs (tokens, latency, model choice, retrieval) without compromising the user experience.
• Translate AI failure patterns, hallucinations, inconsistent outputs, weak reasoning, into clear, prioritized fixes for Product and Engineering.
• Help Product, Growth, and Engineering teams understand where AI changes create real user and business impact.

What We’re Looking For

• 5+ years in software engineering, AI product engineering, ML engineering, or a related technical role, plus 2+ years hands-on with LLMs, prompt engineering, or applied AI systems.
• Proven experience improving AI features post-launch through analytics, experimentation, and structured evaluation, not just initial prompt design.
• Strong grasp of product analytics, funnel metrics, and how to connect AI behavior to business outcomes.
• Ability to diagnose whether an AI quality issue stems from prompts, context, model limitations, UX, or data, and to fix the right layer.
• Strong written communication and the ability to work autonomously in a remote, async environment.
• Comfort working closely with Product, Growth, Design, Engineering, QA, and Analytics.
• Comfort with ownership, transparency, and a results-driven mindset.

Nice to Have

• Experience with A/B testing frameworks, AI observability tools, or custom evaluation pipelines.
• Hands-on work with OpenAI, Anthropic, Gemini, AWS Bedrock, or similar LLM providers.
• Familiarity with RAG, embeddings, semantic search, or tool-calling architectures.
• Background in chatbot, assistant, content generation, or recommendation products.