AI Evaluation Engineer – Business & Operations

About the role

Gramian Consultancy seeks an AI Evaluation Engineer to design and assess realistic business and operational workflow scenarios for advanced AI systems. You will work remotely with a focus on multi‑step decision‑making, process optimization, and operational failure analysis across diverse business environments.

Key responsibilities

• Design realistic business and operational workflow scenarios for AI evaluation.
• Create structured tasks that involve analytics, reporting, and process optimization.
• Develop clear specifications, expected outcomes, and validation logic for each scenario.
• Identify edge cases, bottlenecks, and workflow failure situations.
• Evaluate AI‑generated outputs for reasoning quality, usefulness, and accuracy.
• Collaborate with reviewers and researchers to refine benchmark scenarios.

Required profile

• 3‑10 years of experience in operations, analytics, consulting, or related business systems.
• Strong analytical thinking and operational problem‑solving abilities.
• Hands‑on experience with operational workflows, reporting systems, CRM tools, or business analytics.
• Excellent written communication and documentation skills.

Required skills

• Spreadsheets
• Dashboards
• Operational reporting
• Workflow automation
• CRM tools

What we offer

• Short‑term contract (5 weeks) with full‑time or part‑time options.
• Remote work across multiple time zones.
• Opportunity to shape AI evaluation benchmarks in real‑world business contexts.