APEX-Agents · Law
World425_tas_02
APEX-Agents task World425_tas_02 in AI Agents for Tax Due Diligence. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.
Task prompt
What the agent was asked to do
Review the due diligence file relative to the S-Corp election of Summit Filing Solutions. Determine whether the S-Corporation election was timely made, the date on which it became effective, and any risks arising from the late election. Draft a short memo with your conclusions and include an evaluation of any reasonable cause statement provided for any untimely filing. Send back your memo in a new DOCX file that you create from scratch.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| GPT-5.5showcase | dual | 5/6 | Fail | Share pagePublic trace |
| fireworks models Kimi K2 | dual | 1/6 | Fail | Share pagePublic trace |
| Gemini 3 Flash | dual | 4/6 | Fail | Share pagePublic trace |
| Gemini 3.1 Pro | dual | 2/6 | Fail | Share pagePublic trace |
| GPT-5.4 | dual | 1/6 | Fail | Share pagePublic trace |
| GPT-5.4 mini | dual | 3/6 | Fail | Share pagePublic trace |
| GPT-5.4 nano | dual | 3/6 | Fail | Share pagePublic trace |
Grading rubric
Criteria and grader verdict (showcase run)
States that the S-Corporation election of Summit Filing Solutions, Inc., was effectively made through the filing of Form 2553
PassEvidence: In /root/workspace/filesystem/Memorandum/Summit_S-Corp_Election_Timing_Memo.docx, the memo is titled re “Form 2553 S-corporation election” and concludes “Summit’s Form 2553 was late, but the diligence file supports a working conclusion that the IRS accepted the late election effective January 1, 2019.” Assessment: The criterion asks whether it states the election was effectively made through filing Form 2553. Pass; the memo identifies Form 2553 as the S-election filing and treats IRS acceptance of that filed late election as making the election effective.
States that the S-Corporation election of Summit Filing Solutions, Inc., was effective retroactive to January 1, 2019
PassEvidence: The memo’s executive conclusion states: “If the CP261 notice described in the diligence memo exists and accurately reflects IRS acceptance, Summit’s S election became effective January 1, 2019.” It also says the “apparent effective date is January 1, 2019.” Assessment: The criterion asks whether it states the election was effective retroactive to January 1, 2019. Pass, with a documentation-related qualification that does not negate the stated effective date.
States that under Rev. Proc. 2013-30, an S-Corporation election will be deemed effective if all of the following are true: election filed within 3 years and 75 days of the intended effective date, the applicant’s reasonable cause statement is determined sufficient by the IRS, and the entity shows diligence in correcting the error when discovered
PassEvidence: Under “Rev. Proc. 2013-30 relief,” the memo states the simplified relief procedure generally requires reasonable cause and diligence, and that “Form 2553 was filed within three years and 75 days of the requested effective date, unless…” It also describes reasonable cause and that Summit “acted diligently to correct the mistake after discovery.” Assessment: The criterion asks for the Rev. Proc. 2013-30 conditions: filing within 3 years/75 days, sufficient reasonable cause, and diligence correcting the error. Pass; all required elements are stated, even though the memo also lists additional conditions/exceptions.
States that a sufficient reasonable cause statement was provided by Summit Filing Solutions, Inc., on Form 2553
PassEvidence: The memo’s “Evaluation of the reasonable-cause statement” says “Statement provided. Rev. B states…” and assesses it as “a plausible and relatively strong reasonable-cause narrative.” It concludes that if supporting materials and CP261 exist, “the reasonable-cause position is adequate and likely already accepted by the IRS.” Assessment: The criterion asks whether it states a sufficient reasonable cause statement was provided on Form 2553. Pass; the memo identifies the Form 2553 statement and evaluates it as strong/adequate, while noting documentation gaps.
States that there is residual risk arising from the fact that the file does not contain IRS acknowledgement or acceptance of the late filed Form 2553
PassEvidence: The memo states “the CP261 notice itself was not located” and “Without the CP261 or equivalent IRS transcript, the risk remains meaningful.” Its risk table lists “CP261 / IRS acceptance not in file” as a risk. Assessment: The criterion asks whether it states residual risk from the lack of IRS acknowledgement/acceptance of the late-filed Form 2553. Pass; this risk is explicitly identified.
States that residual risk may include all of the following: C-corporation status for tax years 2019 and 2020, and potential corporate tax and penalties for those years
FailEvidence: The memo states that if acceptance cannot be confirmed, “Summit could face C-corporation treatment for affected years, amended corporate and shareholder returns, corporate-level tax, interest and penalties.” Assessment: The criterion requires residual risk to include all of: C-corporation status for tax years 2019 and 2020, and potential corporate tax and penalties for those years. Fail; although it mentions C-corporation treatment, corporate-level tax, interest and penalties, it does not specifically state that residual C-corporation-status risk applies to both 2019 and 2020.