APEX-Agents · Law
World425_amk_04
APEX-Agents task World425_amk_04 in AI Agents for Tax Due Diligence. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.
Task prompt
What the agent was asked to do
Assume that the Internal Revenue Service denies relief for the transfer of shares to Carrie Canuck, an ineligible shareholder. What amount of taxes (federal plus state) would be owed by Summit Filing Solutions Inc. ("Summit") as a result of the denial for each taxable year, until the earliest time Summit can requalify and file a tax election to become an S corporation, or up to the last year for which we have data, whichever is earlier? - Assume that the federal tax rate is 21% for ordinary business income and the California tax rate is 8.84% for ordinary business income. - Assume that the ordinary business income for each month of the year was exactly 1/12 of the respective year's ordinary business income. - Round values to two decimal places. Print your reply back to me here.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| GPT-5.5showcase | dual | 0/3 | Fail | Share pagePublic trace |
| fireworks models Kimi K2 | dual | 0/3 | Fail | Share pagePublic trace |
| Gemini 3 Flash | dual | 0/3 | Fail | Share pagePublic trace |
| Gemini 3.1 Pro | dual | 0/3 | Fail | Share pagePublic trace |
| GPT-5.4 | dual | 0/3 | Fail | Share pagePublic trace |
| GPT-5.4 mini | dual | 0/3 | Fail | Share pagePublic trace |
| GPT-5.4 nano | dual | 0/3 | Fail | Share pagePublic trace |
Grading rubric
Criteria and grader verdict (showcase run)
States that Summit's tax liability for the 2022 tax year is $41,511.34
FailEvidence: <TEXT_RESPONSE> table states for 2022: “Total federal + state tax” = “$44,265.15.” Assessment: Criterion requires stating Summit's 2022 tax liability is $41,511.34; the response states a different amount, so fail.
States that Summit's tax liability for the 2023 tax year is $59,884.90
FailEvidence: <TEXT_RESPONSE> table states for 2023: “Total federal + state tax” = “$63,857.60.” Assessment: Criterion requires stating Summit's 2023 tax liability is $59,884.90; the response states a different amount, so fail.
States that Summit's tax liability for the 2024 tax year is $93,521.19
FailEvidence: <TEXT_RESPONSE> table states for 2024: “Total federal + state tax” = “$99,725.28.” Assessment: Criterion requires stating Summit's 2024 tax liability is $93,521.19; the response states a different amount, so fail.