APEX-Agents · Law
World425_jcf-03
APEX-Agents task World425_jcf-03 in AI Agents for Tax Due Diligence. Compare dual-harness agent runs across models, scores, and public traces.
Task prompt
What the agent was asked to do
Review Summit's records to determine how Summit can correct any potential questions about the validity of its S corporation tax election. Write a tax memo and put it in a New document (docx) for me to review later.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| Gemini 3 Flash | dual | 7/7 | Pass | Run detailsPublic trace |
| Gemini 3.1 Pro | dual | 6/7 | Fail | Run detailsPublic trace |
| GPT-5.5 | dual | 6/7 | Fail | Run detailsPublic trace |
| GPT-5.4 nano | dual | 5/7 | Fail | Run detailsPublic trace |
| GPT-5.4 | dual | 4/7 | Fail | Run detailsPublic trace |
| GPT-5.4 mini | dual | 4/7 | Fail | Run detailsPublic trace |
| fireworks models Kimi K2 | dual | 3/7 | Fail | Run detailsPublic trace |
Grading rubric
Rubric criteria
Runs are graded against these criteria. Open a run for model-specific verdicts.
States that Summit elected to be taxed as an S corporation
States that the tax code prohibits an S corporation from having a nonresident alien as a shareholder,
States that a transfer of Summit shares to a nonresident alient would automatically terminate Summit's S corporation tax election
States the transfer of Summit shares to Canuck was likely an inadvertent termination of S Corporation status
States that Canuck transferred her Summit shares back to Beaumont within less than a year
States that prompt return of shares to Beaumont may be deemed to satisfy the requirement of prompt corrective action following discovery of an event causing inadvertent termination of a corporation's S Corporation status
States that the only available remedy is for Summit to seek an IRS ruling (private letter ruling) on the validity of its S corporation election