Raycaster / evalsBack to AI Agents for Tax Due Diligence

APEX-Agents · Law

World425_tas_05

5/8Fail

APEX-Agents task World425_tas_05 in AI Agents for Tax Due Diligence. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.

AI Agents for Tax Due DiligenceLaw World 425Dual harnessGrader: rubric
task_ed6f8d835b0141309442d2c373d1c5da
Law World 425
make_new_doc
7 models · dual config

Task prompt

What the agent was asked to do

As you know, Harbor Bridge Private Equity sent initial inquiries to Summit Filing Solutions ("Summit") on matters related to Summit's S-Corp election. Laura Kensington, Summit's Acting CEO, responded with a letter explaining Summit's non-proportionate distributions (there were two instances) and the potential ineligible (non-resident alien) shareholder. She indicated that the shareholders would be willing to make representations and warranties (to be incorporated into the share purchase agreement) on the matters addressed in her letter. Please review the due diligence file and draft the representations and warranties, specifically to address any bad facts and/or identified deficiencies in light of the responses set forth in Ms. Kensington's letter. Reply to me here with your view as a short message.

Published trajectories

Agent runs on this task

Curated dual-harness runs (parsed + original sandbox). Best scored run per model.

ModelHarnessScoreResultLinks
GPT-5.5showcasedual5/8Fail
fireworks models Kimi K2dual6/8Fail
Gemini 3 Flashdual7/8Fail
Gemini 3.1 Produal1/8Fail
GPT-5.4dual2/8Fail
GPT-5.4 minidual2/8Fail
GPT-5.4 nanodual6/8Fail

Grading rubric

Criteria and grader verdict (showcase run)

  1. States the response in the form of a list of representations and warranties by shareholders of Summit Filing Solutions, Inc

    Pass

    Evidence: The response says, “I would include reps substantially as follows:” followed by bullet points including “Shareholder eligibility,” “One class / distributions,” and “No contrary agreements,” and refers to “flat, seller/shareholder-level… reps.” Assessment: The criterion requires the response be in the form of a list of representations and warranties by Summit shareholders; pass because it provides a clear list of proposed seller/shareholder reps.

  2. States that sellers represent and warrant that Summit's S-Corporation election has been valid from January 1, 2019, to date

    Fail

    Evidence: The response discusses preserving S status and states reps about eligible shareholders and distributions, but does not say “Summit’s S-Corporation election has been valid from January 1, 2019, to date” or equivalent. Assessment: The criterion specifically requires a rep as to validity of the S-Corp election for that period; fail because that explicit validity representation is missing.

  3. States that sellers represent and warrant that no nonresident alien has owned Summit stock from January 1, 2019 to date

    Pass

    Evidence: Under “Shareholder eligibility,” the response says “every person who has owned Summit stock since January 1, 2019… was at all times an eligible S-corporation shareholder and not a nonresident alien.” Assessment: This directly states the required seller rep that no nonresident alien owned Summit stock from January 1, 2019 to date; pass.

  4. States that sellers represent and warrant that Carrie Canuck was an eligible S-Corporation shareholder from June 1, 2022, to February 4, 2023

    Pass

    Evidence: The response states “every person who has owned Summit stock since January 1, 2019, including Carrie Canuck during the June 1, 2022-February 4, 2023 period, was at all times an eligible S-corporation shareholder.” Assessment: This matches the required representation that Carrie Canuck was an eligible S-Corporation shareholder during that period; pass.

  5. States that sellers represent and warrant that Carrie Canuck was not a nonresident alien from June 1, 2022, to February 4, 2023

    Pass

    Evidence: The same “Shareholder eligibility” bullet states that Carrie Canuck during “June 1, 2022-February 4, 2023” was “not a nonresident alien.” Assessment: This directly satisfies the required representation; pass.

  6. States that sellers represent and warrant that, since January 1, 2019, Summit has made only distributions pro rata to shareholders in proportion to their respective stock ownership interests

    Pass

    Evidence: The response says, “except for the specifically scheduled May 22, 2022 and February 9, 2024 variances, no distribution differed from stock ownership percentages,” and further characterizes those variances as “not stock distributions.” Assessment: The criterion requires stating that Summit has made only pro rata distributions since January 1, 2019; pass because the response effectively frames all actual stock distributions as pro rata, with the identified variances treated as non-distribution adjustments.

  7. States that sellers represent and warrant that the non-pro rata amount paid to Michael T. Hartwell concurrent with the May 22, 2022, distributions included repayment of his loan to Summit

    Fail

    Evidence: The response states generally that the May 22, 2022 and February 9, 2024 variances were “bona fide documented loan repayments or inter-shareholder purchase-price adjustments.” It does not specifically state that the non-pro rata amount paid to Michael T. Hartwell concurrent with the May 22, 2022 distributions included repayment of his loan to Summit. Assessment: The required specific Hartwell/May 22 loan-repayment representation is not clearly made; fail.

  8. States that the seller represents and warrants that the disproportionate distribution made on February 9, 2024, is based on a written agreement between Mr. Hartwell and Ms. Soto

    Fail

    Evidence: The response refers generally to the February 9, 2024 variance as possibly an “inter-shareholder purchase-price adjustment,” but does not identify a written agreement between Mr. Hartwell and Ms. Soto. Assessment: The criterion requires stating that the February 9, 2024 disproportionate distribution is based on a written Hartwell-Soto agreement; fail because that specific representation is absent.