APEX-Agents · Management Consulting
World 129_CY_Task 5
APEX-Agents task World 129_CY_Task 5 in AI Agents for Legal Contract Review. Compare dual-harness agent runs across models, scores, and public traces.
Task prompt
What the agent was asked to do
Using the latest renewal / churn data (v1.0) and pricing log, identify the average seat changes (increase or decrease) by Brightpath customers from every renewal with seat changes. When doing the calculation, change v4.2 pricing effective start date year to 2023 and make use of monthly price per user to calculate the seat changes. Present your result, printing it right in here to the nearest 0.01.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| GPT-5.4 mini | dual | 1/1 | Pass | Run detailsPublic trace |
| GPT-5.4 nano | dual | 1/1 | Pass | Run detailsPublic trace |
| GPT-5.5 | dual | 1/1 | Pass | Run detailsPublic trace |
| Gemini 3.1 Pro | dual | 0/1 | Fail | Run detailsPublic trace |
| GPT-5.4 | dual | 0/1 | Fail | Run detailsPublic trace |
Grading rubric
Rubric criteria
Runs are graded against these criteria. Open a run for model-specific verdicts.
States that the average seat change by Brightpath customers across all contract renewals with seat changes is -19.02 seats