APEX-Agents · Law
Task awys8050
APEX-Agents task Task awys8050 in AI Agents for SEC Disclosure Analysis. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.
Task prompt
What the agent was asked to do
CrowdStrike's general counsel sent us a complaint filed in U.S. district court by Larry Stone, alleging violations under Sections 10(b) and 20(a) of the Securities Exchange Act of 1934, as well as Rule 10b-5 arising from false statements or omissions regarding its Falcon Sensor, the update of which causing the widely-reported July, 2024 service outage, leading to his Class A stock suffering a considerable loss in value. Review our directories and the attached file for analysis and reply back to me with a short memo in a new dox file. Determine whether the Plymouth matter's class, which is pending certification and does not show a related opt out, is likely to support a successful motion to dismiss Stone's suit.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| GPT-5.5showcase | dual | 6/8 | Fail | Share pagePublic trace |
| fireworks models Kimi K2 | dual | 5/8 | Fail | Share pagePublic trace |
| Gemini 3 Flash | dual | 8/8 | Pass | Share pagePublic trace |
| Gemini 3.1 Pro | dual | 3/8 | Fail | Share pagePublic trace |
| GPT-5.4 | dual | 0/8 | Fail | Share pagePublic trace |
| GPT-5.4 mini | dual | 1/8 | Fail | Share pagePublic trace |
| GPT-5.4 nano | dual | 7/8 | Fail | Share pagePublic trace |
Grading rubric
Criteria and grader verdict (showcase run)
States that Larry purchased 56 shares
PassEvidence: In `/root/workspace/filesystem/Stone_Plymouth_Dismissal_Memo.docx`, Key facts reviewed states: “Larry Stone as purchasing 56 CrowdStrike Class A shares on April 3, 2024.” Assessment: The criterion asks whether it states Larry purchased 56 shares; pass because the memo explicitly states 56 shares.
States that Larry purchased Class A shares
PassEvidence: `/root/workspace/filesystem/Stone_Plymouth_Dismissal_Memo.docx`, Key facts reviewed: “Larry Stone as purchasing 56 CrowdStrike Class A shares on April 3, 2024.” Assessment: The criterion asks whether it states Larry purchased Class A shares; pass because the memo explicitly identifies the shares as CrowdStrike Class A shares.
States that Larry purchased shares on April 3, 2024
PassEvidence: `/root/workspace/filesystem/Stone_Plymouth_Dismissal_Memo.docx`, Key facts reviewed: “Larry Stone as purchasing 56 CrowdStrike Class A shares on April 3, 2024.” Assessment: The criterion asks whether it states Larry purchased shares on April 3, 2024; pass because the memo gives that exact date.
States that class action member plaintiffs are those who purchased Class A shares of CrowdStrike stock between November 29, 2023 and July 29, 2024
PassEvidence: `/root/workspace/filesystem/Stone_Plymouth_Dismissal_Memo.docx`, Key facts reviewed: “The original Plymouth complaint asserted... claims for purchasers of CrowdStrike Class A common stock from November 29, 2023 through July 29, 2024.” Assessment: The criterion asks whether it states class action member plaintiffs are those who purchased Class A shares between November 29, 2023 and July 29, 2024; pass because the original Plymouth class scope is stated with those securities and dates.
States that Larry is a member of the class
PassEvidence: `/root/workspace/filesystem/Stone_Plymouth_Dismissal_Memo.docx`, Key facts reviewed states Stone’s purchase “falls within both the original Plymouth period and the consolidated class period,” and Bottom line refers to Stone being “likely within the pending Plymouth/In re CrowdStrike putative class.” Assessment: The criterion asks whether it states Larry is a member of the class; pass because the memo conveys Stone falls within/is likely within the putative class.
States that Larry did not opt out of the class
PassEvidence: `/root/workspace/filesystem/Stone_Plymouth_Dismissal_Memo.docx`, Key facts reviewed: “No class-certification order, Rule 23(b)(3) class notice... or Stone opt-out record was located”; Bottom line says “the file does not show a Stone opt-out opportunity.” Assessment: The criterion asks whether it states Larry did not opt out of the class; pass because the memo states no Stone opt-out record/opportunity was found.
States that the suit can be dismissed
FailEvidence: `/root/workspace/filesystem/Stone_Plymouth_Dismissal_Memo.docx`, Bottom line: “A motion to dismiss Stone solely because he is likely within the pending... class is unlikely to succeed”; Conclusion: “not likely by itself to support a successful motion to dismiss Stone’s suit.” Assessment: The criterion asks whether it states the suit can be dismissed; fail because the memo states the opposite as to dismissal based on Plymouth/class membership, recommending transfer/consolidation/stay instead.
States that the grounds for dismissal are under the federal claim-splitting/duplicative-litigation doctrine
FailEvidence: `/root/workspace/filesystem/Stone_Plymouth_Dismissal_Memo.docx`, Legal analysis section heading says “Preclusion and claim-splitting arguments are premature,” and explains likely remedies are “a stay, transfer, consolidation, or management order—not Rule 12 dismissal with prejudice.” Assessment: The criterion asks whether it states the grounds for dismissal are under the federal claim-splitting/duplicative-litigation doctrine; fail because the memo does not assert that doctrine as grounds for dismissal and instead says claim-splitting/duplicative-litigation dismissal is premature/unlikely.