APEX-Agents · Management Consulting
world130_HO_04
APEX-Agents task world130_HO_04 in AI Agents for Employment Law Analysis. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.
Task prompt
What the agent was asked to do
I want to quantify the average annual productivity loss at a cost level for each employee in each primary role based on the sum of average hours spent doing manual entry, searching data, and fixing errors. Then, I want to calculate the total productivity loss cost HarFeast faces every year, company-wide. Note that the survey responses represent one week of work. Report your final answer as a message here, rounded to the nearest dollar.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| GPT-5.5showcase | dual | 0/9 | Fail | Share pagePublic trace |
| Gemini 3.1 Pro | dual | 0/9 | Fail | Share pagePublic trace |
| GPT-5.4 | dual | 0/9 | Fail | Share pagePublic trace |
| GPT-5.4 mini | dual | 9/9 | Pass | Share pagePublic trace |
| GPT-5.4 nano | dual | 0/9 | Fail | Share pagePublic trace |
Grading rubric
Criteria and grader verdict (showcase run)
States the average annual productivity loss cost of an Administrative/Support Staff employee is $6,232
FailEvidence: TEXT_RESPONSE table states Administrative/Support Staff annual loss/employee is "$12,464". Assessment: Criterion requires stating "$6,232"; fail because the stated value is different.
States the average annual productivity loss cost of a Demand Planning/Forecasting employee is $9,504
FailEvidence: TEXT_RESPONSE table states Demand Planning/Forecasting annual loss/employee is "$19,315". Assessment: Criterion requires stating "$9,504"; fail because the stated value is different.
States the average annual productivity loss cost of a Maintenance Technician employee is $8,088
FailEvidence: TEXT_RESPONSE table states Maintenance Technician annual loss/employee is "$15,935". Assessment: Criterion requires stating "$8,088"; fail because the stated value is different.
States the average annual productivity loss cost of a Plant Management employee is $12,563
FailEvidence: TEXT_RESPONSE table states Plant Management annual loss/employee is "$25,893". Assessment: Criterion requires stating "$12,563"; fail because the stated value is different.
States the average annual productivity loss cost of a Production Supervisor/Team Lead employee is $8,549
FailEvidence: TEXT_RESPONSE table states Production Supervisor/Team Lead annual loss/employee is "$17,709". Assessment: Criterion requires stating "$8,549"; fail because the stated value is different.
States the average annual productivity loss cost of a Production/Manufacturing Operator employee is $5,767
FailEvidence: TEXT_RESPONSE table states Production/Manufacturing Operator annual loss/employee is "$11,861". Assessment: Criterion requires stating "$5,767"; fail because the stated value is different.
States the average annual productivity loss cost of a Quality Control/Quality Assurance employee is $7,359
FailEvidence: TEXT_RESPONSE table states Quality Control/Quality Assurance annual loss/employee is "$16,415". Assessment: Criterion requires stating "$7,359"; fail because the stated value is different.
States the average annual productivity loss cost of a Supply Chain/Logistics Coordinator is $8,189
FailEvidence: TEXT_RESPONSE table states Supply Chain/Logistics Coordinator annual loss/employee is "$14,474". Assessment: Criterion requires stating "$8,189"; fail because the stated value is different.
States the total annual productivity loss cost is $12,622,039
FailEvidence: TEXT_RESPONSE states "Total annual company-wide productivity loss cost: $43,761,548." Assessment: Criterion requires stating total annual productivity loss cost is "$12,622,039"; fail because the stated total is different.