AI-Assisted Workflow Evaluator (Contract)

About Us

Gramian Consultancy is a boutique consultancy specializing in IT professional services and engineering talent solutions. With a strong background in software engineering and leadership, we help companies build high-performing teams by matching them with professionals who truly fit their needs.

Role Overview

We are looking for experienced professionals from software engineering, DevOps, SRE, cloud, platform, or infrastructure engineering backgrounds to test and evaluate AI-assisted workflows across modern developer and workplace tools. The role focuses on assessing technical accuracy, workflow realism, command and configuration quality, platform interactions, and practical usability. No prior AI evaluation experience is required, but hands-on use of AI coding or productivity agents is highly relevant.

Responsibilities

• Test and validate AI-assisted developer and infrastructure workflows.
• Evaluate commands, configurations, and outputs across cloud and platform engineering scenarios.
• Review workflows involving GitHub, GitLab, JIRA, AWS, Azure, GCP, Kubernetes, Docker, and Terraform.
• Identify technical inaccuracies, broken steps, unrealistic assumptions, or workflow gaps.
• Assess how AI agents interact with connected workplace tools such as Slack, Google Drive, and Microsoft 365.
• Document observations and provide clear, actionable technical feedback.
• Recommend workflow improvements based on real-world engineering standards.
• Participate in asynchronous review and virtual discussions to clarify task requirements.
• Evaluate whether AI-generated solutions are practical, reproducible, and technically sound.
• Maintain clear documentation across all submitted evaluations.

Requirements

• Professional experience in software engineering, DevOps, SRE, cloud, platform, or infrastructure engineering.
• Hands-on experience with at least one major cloud platform: AWS, Azure, or GCP.
• Practical experience with Kubernetes, Docker, Terraform, or comparable infrastructure tooling.
• Familiarity with developer platforms such as GitHub, GitLab, or JIRA.
• Experience with CI/CD pipelines and modern infrastructure or platform workflows.
• Professional experience using AI agents such as ChatGPT, Claude Code, Claude Cowork, or Codex.
• Ability to evaluate technical workflows and identify incorrect or inefficient implementations.
• Strong written and spoken English.