APEX-Agents · Management Consulting

World 129_CY_Task 5

1/1Pass

APEX-Agents task World 129_CY_Task 5 in AI Agents for Legal Contract Review. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.

AI Agents for Legal Contract ReviewManagement Consulting World 129Dual harnessGrader: rubric

Task IDtask_1e19e170dbac46ae98a24930a70b4b73

WorldManagement Consulting World 129

Expected outputmessage_in_console

Published runs5 models · dual config

Task prompt

What the agent was asked to do

Using the latest renewal / churn data (v1.0) and pricing log, identify the average seat changes (increase or decrease) by Brightpath customers from every renewal with seat changes. When doing the calculation, change v4.2 pricing effective start date year to 2023 and make use of monthly price per user to calculate the seat changes. Present your result, printing it right in here to the nearest 0.01.

Published trajectories

Agent runs on this task

Curated dual-harness runs (parsed + original sandbox). Best scored run per model.

Model	Harness	Score	Result	Links
GPT-5.5showcase	dual	1/1	Pass	Share page Public trace
Gemini 3.1 Pro	dual	0/1	Fail	Share page Public trace
GPT-5.4	dual	0/1	Fail	Share page Public trace
GPT-5.4 mini	dual	1/1	Pass	Share page Public trace
GPT-5.4 nano	dual	1/1	Pass	Share page Public trace

Grading rubric

Criteria and grader verdict (showcase run)

States that the average seat change by Brightpath customers across all contract renewals with seat changes is -19.02 seats
Pass
Evidence: The response states, “Average signed seat change: -19.02 seats” and clarifies it is “across renewals marked as seat expanded/reduced.” Assessment: The criterion requires stating that the average seat change by Brightpath customers across all contract renewals with seat changes is -19.02 seats; this is clearly stated. Pass.