AI Benchmark Engineer – Data Analysis

About the role

Turing is looking for an experienced AI Benchmark Engineer to design and develop multi‑agent benchmark tasks that evaluate advanced AI systems on complex data‑analysis workflows. You will create realistic datasets, define verification logic, and build reproducible evaluation environments.

Key responsibilities

• Design and author multi‑agent benchmark tasks focused on analytical reasoning, coordination, and execution.
• Create synthetic or curated datasets from domains such as finance, operations, security, or market analysis.
• Develop tasks that require cross‑referencing, anomaly detection, contradiction identification, and statistical computation across multiple sources.
• Write decomposition guides that split work among specialist sub‑agents (e.g., financial, technical, security analysts).
• Implement precise oracle logic or verification scripts to validate specific analytical conclusions.
• Build reproducible evaluation environments using Python and Docker.
• Review performance signals and refine tasks to improve determinism, clarity, difficulty, and scoring quality.

Required profile

• 5+ years of experience in data analysis.
• Strong proficiency in SQL and Python (pandas, NumPy or similar).
• Hands‑on experience with real‑world, messy datasets (CSV, JSON, logs, reports).
• Ability to design non‑trivial analytical questions with clear, verifiable answers.
• Solid understanding of statistical concepts such as averages, distributions, outliers, and correlations.
• Familiarity with AI coding benchmark environments (e.g., SWE‑bench, Terminal‑Bench).
• Comfortable working with Docker (writing Dockerfiles, building images, debugging containers).

Required skills

• Python
• SQL
• pandas
• NumPy
• Docker

What we offer

• Work on cutting‑edge AI projects with leading foundation‑model companies.
• Collaborate on high‑impact, mission‑critical AI systems.