Raycaster / evalsBack to AI Agents for Tax Due Diligence

APEX-Agents · Law

World425_jcf_01

6/9Fail

APEX-Agents task World425_jcf_01 in AI Agents for Tax Due Diligence. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.

AI Agents for Tax Due DiligenceLaw World 425Dual harnessGrader: rubric
task_8705d28530a94c2880fbfd7190e257d4
Law World 425
edit_existing_doc
7 models · dual config

Task prompt

What the agent was asked to do

Revise the Stock Purchase Agreement. I want you to edit the existing file. I want you to protect the Buyer given a potential $399,540 tax exposure tied to Summit’s potentially invalid S-Corp election. Update the agreement to include strong seller reps, warranties, covenants, and indemnities to protect the buyer, as well as tax-related pre-closing and post-closing obligations so that any liabilities associated with the S-Corporation issue are borne by the Sellers.

Published trajectories

Agent runs on this task

Curated dual-harness runs (parsed + original sandbox). Best scored run per model.

ModelHarnessScoreResultLinks
GPT-5.5showcasedual6/9Fail
fireworks models Kimi K2dual6/9Fail
Gemini 3 Flashdual7/9Fail
Gemini 3.1 Produal5/9Fail
GPT-5.4dual6/9Fail
GPT-5.4 minidual7/9Fail
GPT-5.4 nanodual7/9Fail

Grading rubric

Criteria and grader verdict (showcase run)

  1. States that Summit has retained valid S Corporation status since 1/1/2019

    Pass

    Evidence: /root/workspace/.../Stock Purchase Agmt v1.docx, Section 3.11(f)(i): “valid election to be treated as an S corporation… effective January 1, 2019” and “has remained continuously valid and in effect at all times through and including the Closing Date.” Assessment: The criterion requires stating Summit retained valid S Corporation status since 1/1/2019; pass.

  2. States that Summit has not taken any action that would disqualify it as an S corporation

    Pass

    Evidence: Section 3.11(f)(iii): “The Company has not issued, authorized, or been party to any… arrangement that… causes or could cause a termination or invalidity of the Company’s S-corporation election.” Section 5.10(a) also requires Sellers to “take no action… that would cause or could cause any S-Corp Tax Losses.” Assessment: The criterion requires stating Summit has not taken disqualifying action; pass.

  3. States that the Disclosure Letter sets forth a true and complete copy of the IRS Form 2553 filed by the Company

    Fail

    Evidence: Section 5.10(b) requires delivery of “complete and correct copies of all Records… including Form 2553”; Section 3.11(f)(vi) says Sellers delivered “complete and correct copies of all federal, state, and local S-corporation election filings.” Assessment: The criterion specifically requires stating the Disclosure Letter sets forth a true and complete copy of IRS Form 2553 filed by the Company. The file mentions delivery of Form 2553/election filings, but not that the Disclosure Letter sets it forth. Fail.

  4. States that all tax returns required to be filed by or on behalf of Summit have been timely filed

    Pass

    Evidence: Section 3.11(a)(i): “Each Acquired Company has filed or caused to be filed on a timely basis all Tax Returns that were required to be filed by or with respect to it.” Assessment: The criterion requires stating all required Summit tax returns have been timely filed; pass.

  5. States that all taxes shown to be due on tax returns required to be filed by or on behalf of Summit have been timely paid

    Fail

    Evidence: Section 3.11(a)(iv): “Each Acquired Company has paid, or made provision for the payment of, all Taxes that have or could have become due for all periods covered by any Tax Return or otherwise…” Assessment: The criterion requires stating all taxes shown due on required tax returns have been timely paid. The agreement states taxes due have been paid/provided for, but does not clearly state taxes shown due on returns were timely paid. Fail.

  6. States that Sellers shall be responsible for full payment of any liabilities pertaining to the Seller’s failure to retain a valid S Corporation status

    Pass

    Evidence: Definition of “S-Corp Tax Losses” includes Losses attributable to “any failure of the Company to qualify as, or to maintain continuously valid status as, an S corporation…” Section 11.2(g) requires Sellers to indemnify for “any S-Corp Tax Losses”; Section 7.5(a) says Sellers “pay or reimburse… all S-Corp Tax Losses.” Assessment: The criterion requires Sellers be responsible for full payment of liabilities from failure to retain valid S status; pass.

  7. States in the pre-closing obligations that Sellers shall provide Buyer with a tax clearance certificate(s) for each jurisdiction where Summit is organized or qualified to do business; and (2) that Sellers shall provide Buyer with all of Summit’s tax returns if not previously provided

    Fail

    Evidence: Section 5.10(b) requires Sellers to deliver “Forms 1120-S… Tax Returns” and related records. Section 2.4(a)(iv) requires good standing certificates “and payment of applicable state Taxes,” but not tax clearance certificates. Assessment: The criterion is conjunctive: pre-closing obligations must require tax clearance certificates for each organized/qualified jurisdiction and all tax returns if not previously provided. Tax returns are addressed, but tax clearance certificates are not. Fail.

  8. States at least one of the following post-closing obligations: (1) that Sellers shall cooperate with Buyer in matters pertaining to Summit’s failure to retain valid S Corporation status; and (2) that Sellers shall cooperate with Buyer in matters pertaining to Summit’s failure to satisfy pre-closing tax obligations

    Pass

    Evidence: Section 7.5(c): “Sellers shall… cooperate fully with Buyer… in… any S-Corp Relief Request… or other document or action… necessary or desirable to address… any S-Corp Tax Losses.” Section 11.11(c) similarly requires full cooperation in Tax Proceedings/S-Corp Relief Requests. Assessment: The criterion requires at least one listed post-closing cooperation obligation; the S-corporation cooperation obligation is stated. Pass.

  9. States that Sellers indemnify the Buyer from any liability that the Buyer suffers arising from Summit’s failure to retain a valid S Corporation status

    Pass

    Evidence: Section 11.2: “Sellers, jointly and severally, shall indemnify and hold harmless Buyer…” including subsection (g) “any S-Corp Tax Losses.” The definition of S-Corp Tax Losses includes losses arising from failure to qualify as or maintain S-corporation status. Assessment: The criterion requires Sellers indemnify Buyer for liability from Summit’s failure to retain valid S status; pass.