APEX-Agents · Law
World425_jrf_01
APEX-Agents task World425_jrf_01 in AI Agents for Tax Due Diligence. Compare dual-harness agent runs across models, scores, and public traces.
Task prompt
What the agent was asked to do
We need to review Summit's historical distributions. Write me a memo analyzing whether Summit's shareholder distributions are in accordance with US tax code. Cite the exact tax code in each instance (short citations are acceptable). Reply to me right in here please.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| Gemini 3 Flash | dual | 3/3 | Pass | Run detailsPublic trace |
| Gemini 3.1 Pro | dual | 3/3 | Pass | Run detailsPublic trace |
| GPT-5.4 | dual | 3/3 | Pass | Run detailsPublic trace |
| GPT-5.5 | dual | 3/3 | Pass | Run detailsPublic trace |
| fireworks models Kimi K2 | dual | 2/3 | Fail | Run detailsPublic trace |
| GPT-5.4 mini | dual | 2/3 | Fail | Run detailsPublic trace |
| GPT-5.4 nano | dual | 1/3 | Fail | Run detailsPublic trace |
Grading rubric
Rubric criteria
Runs are graded against these criteria. Open a run for model-specific verdicts.
States that Summit's shareholder distributions are disproportionate
States that Summit's disproportionate distributions violate US tax code
States that a nonresident alien receiving shares violates tax code