Senior Product Engineer (Agent Training and Evals)
Does Prolific sponsor H-1B visas?
Check real filing data, salaries & approval rates on VisaTalentUSA
Job Description
Remote AI and machine learning roles on HireSkys range from AI engineering and NLP to computer vision and prompt engineering. Listings usually name their frameworks and models — match them against your own hands-on work before applying.
Key Skills Required
Master these to land this role
Want to know if you're a match for this job?
Prolific is not just another player in the AI space – we are the architects of the human data infrastructure that’s reshaping the landscape of AI development. In a world where foundational AI technologies are increasingly commoditized, it’s the quality and diversity of human-generated data that truly differentiates products and models.
As a Senior Product Engineer, you’ll be one of the early engineers in our Agentic Training & Evals team.
You’ll build the infrastructure that lets us evaluate AI agents, capturing agent traces at scale alongside real human expert work in safe and secure environments.
This is a genuine 0→1 role. There isn’t a well-established playbook or existing architecture for you to simply maintain. You’ll have significant influence and ownership over the technical direction of the team and the products we build.
You’ll work closely with engineering, product, and AI research to turn ambiguous problems into scalable systems, and you’ll see the impact of what you build directly.
What you’ll bring to the role:
- Strong experience building scalable infrastructure or distributed systems in the cloud
- Solid understanding of how systems are deployed, scaled, and secured
- Hands-on experience with integrating and evaluating AI models or agents
- Interest in how AI agents are trained, evaluated, and improved
- Bonus: experience with image, video, or audio processing
- Strong product mindset with a focus on delivering business and customer value
- Comfort with ambiguity, and the ability to move quickly from idea to working system
- Strong communication and collaboration skills
- Python experience is useful, but not essential
What you’ll be doing in the role:
- Build tooling for capturing and processing data from agents and humans performing real-world tasks, at significant scale
- Solve hard problems around compute, orchestration, scaling, security, and reliability
- Help develop approaches for training, benchmarking, and evaluating AI agents
- Work with emerging agentic AI technologies and rapidly evolving models
- Make pragmatic technical decisions in a highly ambiguous environment
- Partner closely with product engineering and AI research to turn customer problems into working products
- Drive direct impact in a fast-moving, high-ownership, cross-functional team
Key Technologies:
- Cloud Platforms: Google Cloud Platform
- Programming Languages: Python, JavaScript, and TypeScript
- Frameworks: FastAPI, Vue.js, Container-based and Serverless architectures
- Databases: Postgres SQL
- DevOps and Monitoring: GitHub Actions, Kubernetes, DataDog
How would you rate this job post?
See what other professionals think about this role.
How applying works on HireSkys
- 1.Click Apply. You go to the application — the employer's own page or form, or the HireSkys apply flow — and submit your profile as the employer asks.
- 2.The employer reviews. Shortlisting is done by the employer, not by HireSkys, so the timeline and next steps come from them.
- 3.Hear back directly. If shortlisted, the employer contacts you directly. Applying through HireSkys is free for jobseekers — never pay for a job application.
Similar Opportunities
More Openings at Prolific
Prolific
View Company ProfileProlific is a highly disruptive, Series A data research technology company fundamentally designed to build the largest and highest-quality pool of human-derived data in the world. Founded in 2014 by Phelim Bradley at Oxford University and headquartered in London, United Kingdom, the platform completely revolutionizes how organizations conduct online experiments, surveys, and AI training. Under the hood, Prolific seamlessly connects over 35,000 data collectors with an elite, actively vetted network of over 200,000 verified human participants across 80+ languages—launching a new study every two minutes. Their primary target audience spans cutting-edge AI developers needing precise RLHF (Reinforcement Learning from Human Feedback) and model evaluation data, Ivy League academic researchers, and enterprise UX teams who desperately need to bypass the fraud, bots, and noise of traditional panel providers. What sets Prolific apart in the fiercely competitive market research landscape is its uncompromising commitment to data integrity and ethical compensation; leveraging an advanced, API-first infrastructure and proprietary anti-bot protocols to deliver high-fidelity insights and specialized domain expertise in hours, not weeks.
Safety First
- Never pay for a job application.
- Do not share sensitive bank info.
- Verify the client before starting work.




