Raycaster / evalsBack to AI Agents for Tax Due Diligence

APEX-Agents · Law

World425_tas_02

5/6Fail

APEX-Agents task World425_tas_02 in AI Agents for Tax Due Diligence. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.

AI Agents for Tax Due DiligenceLaw World 425Dual harnessGrader: rubric
task_ce4a398d9cf64e63aa54cb88b6615c93
Law World 425
make_new_doc
7 models · dual config

Task prompt

What the agent was asked to do

Review the due diligence file relative to the S-Corp election of Summit Filing Solutions. Determine whether the S-Corporation election was timely made, the date on which it became effective, and any risks arising from the late election. Draft a short memo with your conclusions and include an evaluation of any reasonable cause statement provided for any untimely filing. Send back your memo in a new DOCX file that you create from scratch.

Published trajectories

Agent runs on this task

Curated dual-harness runs (parsed + original sandbox). Best scored run per model.

ModelHarnessScoreResultLinks
GPT-5.5showcasedual5/6Fail
fireworks models Kimi K2dual1/6Fail
Gemini 3 Flashdual4/6Fail
Gemini 3.1 Produal2/6Fail
GPT-5.4dual1/6Fail
GPT-5.4 minidual3/6Fail
GPT-5.4 nanodual3/6Fail

Grading rubric

Criteria and grader verdict (showcase run)

  1. States that the S-Corporation election of Summit Filing Solutions, Inc., was effectively made through the filing of Form 2553

    Pass

    Evidence: In /root/workspace/filesystem/Memorandum/Summit_S-Corp_Election_Timing_Memo.docx, the memo is titled re “Form 2553 S-corporation election” and concludes “Summit’s Form 2553 was late, but the diligence file supports a working conclusion that the IRS accepted the late election effective January 1, 2019.” Assessment: The criterion asks whether it states the election was effectively made through filing Form 2553. Pass; the memo identifies Form 2553 as the S-election filing and treats IRS acceptance of that filed late election as making the election effective.

  2. States that the S-Corporation election of Summit Filing Solutions, Inc., was effective retroactive to January 1, 2019

    Pass

    Evidence: The memo’s executive conclusion states: “If the CP261 notice described in the diligence memo exists and accurately reflects IRS acceptance, Summit’s S election became effective January 1, 2019.” It also says the “apparent effective date is January 1, 2019.” Assessment: The criterion asks whether it states the election was effective retroactive to January 1, 2019. Pass, with a documentation-related qualification that does not negate the stated effective date.

  3. States that under Rev. Proc. 2013-30, an S-Corporation election will be deemed effective if all of the following are true: election filed within 3 years and 75 days of the intended effective date, the applicant’s reasonable cause statement is determined sufficient by the IRS, and the entity shows diligence in correcting the error when discovered

    Pass

    Evidence: Under “Rev. Proc. 2013-30 relief,” the memo states the simplified relief procedure generally requires reasonable cause and diligence, and that “Form 2553 was filed within three years and 75 days of the requested effective date, unless…” It also describes reasonable cause and that Summit “acted diligently to correct the mistake after discovery.” Assessment: The criterion asks for the Rev. Proc. 2013-30 conditions: filing within 3 years/75 days, sufficient reasonable cause, and diligence correcting the error. Pass; all required elements are stated, even though the memo also lists additional conditions/exceptions.

  4. States that a sufficient reasonable cause statement was provided by Summit Filing Solutions, Inc., on Form 2553

    Pass

    Evidence: The memo’s “Evaluation of the reasonable-cause statement” says “Statement provided. Rev. B states…” and assesses it as “a plausible and relatively strong reasonable-cause narrative.” It concludes that if supporting materials and CP261 exist, “the reasonable-cause position is adequate and likely already accepted by the IRS.” Assessment: The criterion asks whether it states a sufficient reasonable cause statement was provided on Form 2553. Pass; the memo identifies the Form 2553 statement and evaluates it as strong/adequate, while noting documentation gaps.

  5. States that there is residual risk arising from the fact that the file does not contain IRS acknowledgement or acceptance of the late filed Form 2553

    Pass

    Evidence: The memo states “the CP261 notice itself was not located” and “Without the CP261 or equivalent IRS transcript, the risk remains meaningful.” Its risk table lists “CP261 / IRS acceptance not in file” as a risk. Assessment: The criterion asks whether it states residual risk from the lack of IRS acknowledgement/acceptance of the late-filed Form 2553. Pass; this risk is explicitly identified.

  6. States that residual risk may include all of the following: C-corporation status for tax years 2019 and 2020, and potential corporate tax and penalties for those years

    Fail

    Evidence: The memo states that if acceptance cannot be confirmed, “Summit could face C-corporation treatment for affected years, amended corporate and shareholder returns, corporate-level tax, interest and penalties.” Assessment: The criterion requires residual risk to include all of: C-corporation status for tax years 2019 and 2020, and potential corporate tax and penalties for those years. Fail; although it mentions C-corporation treatment, corporate-level tax, interest and penalties, it does not specifically state that residual C-corporation-status risk applies to both 2019 and 2020.