APEX-Agents · Law
World425_amk_01
APEX-Agents task World425_amk_01 in AI Agents for Tax Due Diligence. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.
Task prompt
What the agent was asked to do
Summit filed Form 2553 (Rev B) with the Internal Revenue Service ("IRS") on February 19, 2019. Do you see any potential problems under Treasury Regulation 1.1362-6(b)? Give me your analysis back here so I can decide what to do.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| GPT-5.5showcase | dual | 6/6 | Pass | Share pagePublic trace |
| fireworks models Kimi K2 | dual | 0/6 | Fail | Share pagePublic trace |
| Gemini 3 Flash | dual | 6/6 | Pass | Share pagePublic trace |
| Gemini 3.1 Pro | dual | 6/6 | Pass | Share pagePublic trace |
| GPT-5.4 | dual | 1/6 | Fail | Share pagePublic trace |
| GPT-5.4 mini | dual | 1/6 | Fail | Share pagePublic trace |
| GPT-5.4 nano | dual | 1/6 | Fail | Share pagePublic trace |
Grading rubric
Criteria and grader verdict (showcase run)
States that Treasury Regulation 1.1362-6(b)(2)(i) requires that when the stock of the corporation is owned by husband and wife as community property, each spouse must consent to the election
PassEvidence: Response states, “Under § 1.1362-6(b)(2)(i), if Gregory was married and the stock or income from the stock was community property, his spouse also needed to consent,” and earlier quotes that “each person having a community interest... must consent.” Assessment: The criterion asks whether it states the regulation requires each spouse to consent when stock is owned by husband and wife as community property. This is clearly conveyed. Pass.
States that Wisconsin is a community property state
PassEvidence: Response states, “Wisconsin is a community-property state for federal community-property purposes,” and notes the IRS publication list includes “Wisconsin.” Assessment: The criterion asks whether it states Wisconsin is a community property state. Pass.
States that IRS Form 2553 provides that shareholder Gregory Beaumont resides in a community property state
PassEvidence: Response states, “Gregory Beaumont is listed at a Wisconsin address on the shareholder records and the Form 2553 materials” and then discusses Wisconsin as a community-property state. Assessment: The criterion asks whether it states IRS Form 2553 provides Gregory Beaumont resides in a community property state. The response specifically ties Gregory’s Wisconsin address to Form 2553 materials, satisfying this chat-coverage item. Pass.
States that Gregory Beaumont's marital status is unclear as of February 19, 2019
PassEvidence: Response states, “This is contingent: if Gregory was unmarried... there may be no issue” and recommends verifying “marital status... as of the election date.” It also says “if Gregory was married...” rather than asserting he was. Assessment: The criterion asks whether it states Gregory Beaumont's marital status is unclear as of February 19, 2019. Although not using the word “unclear,” the response clearly conveys uncertainty and need to verify as of the election date. Pass.
States that if Gregory Beaumont was married as of February 19, 2019, a consent from his spouse is potentially missing
PassEvidence: Response states, “if Gregory was married and the stock or income from the stock was community property, his spouse also needed to consent. I did not see a spouse consent on Rev B.” It summarizes “possible missing community-property spouse consent for Gregory Beaumont due to Wisconsin.” Assessment: The criterion asks whether it states that if Gregory was married as of February 19, 2019, spouse consent is potentially missing. This is clearly stated with the relevant contingency. Pass.
States that Gregory Beaumont's marital status at the time of signing Form 2553, February 19, 2019 should be confirmed
PassEvidence: Response recommends verifying “marital status, domicile, and property characterization as of the election date” and in next steps: “Marital/community-property confirmations for each 2019 shareholder, especially Gregory Beaumont.” Assessment: The criterion asks whether it states Gregory’s marital status at the time of signing Form 2553, February 19, 2019 should be confirmed. The response says to verify his marital status as of the election date/2019 shareholder context. Pass.