APEX-Agents · Law
World415_DM_01
APEX-Agents task World415_DM_01 in AI Agents for SEC Disclosure Analysis. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.
Task prompt
What the agent was asked to do
MLT is a CrowdStrike customer severely affected by the outage. MLT filed a lawsuit for negligence seeking to recover damages arising from the outage. Additionally, MLT successfully transferred venue to Georgia. CrowdStrike is considering moving for summary judgment on the basis that the outage does not rise to the level of gross negligence and the exculpatory clause in the contract applies (the "Motion"). Can you tell me if CrowdStrike is likely to succeed in its Motion? Reply to me with your view, giving me a Yes or No and a short explanation.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| GPT-5.5showcase | dual | 3/3 | Pass | Share pagePublic trace |
| fireworks models Kimi K2 | dual | 3/3 | Pass | Share pagePublic trace |
| Gemini 3 Flash | dual | 3/3 | Pass | Share pagePublic trace |
| Gemini 3.1 Pro | dual | 3/3 | Pass | Share pagePublic trace |
| GPT-5.4 | dual | 3/3 | Pass | Share pagePublic trace |
| GPT-5.4 mini | dual | 3/3 | Pass | Share pagePublic trace |
| GPT-5.4 nano | dual | 3/3 | Pass | Share pagePublic trace |
Grading rubric
Criteria and grader verdict (showcase run)
States No, CrowdStrike is unlikely to succeed in its Motion
PassEvidence: The response begins: “**No.** … I would not expect CrowdStrike to win summary judgment outright.” Assessment: The criterion asks whether it “States No, CrowdStrike is unlikely to succeed in its Motion.” Pass; the response clearly gives No and says CrowdStrike likely will not win summary judgment outright.
States that the determination of whether the degree of negligence amounts to gross negligence is typically a question of fact for a jury
PassEvidence: The response says “the outage facts likely create a jury issue” and that the Georgia court found allegations “sufficient to state a claim for gross negligence.” Assessment: The criterion asks whether it states that whether negligence amounts to gross negligence is typically a question of fact for a jury. Pass; “likely create a jury issue” conveys that this determination is for the jury/factfinder, in the context of gross negligence.
States that gross negligence cannot be waived by an exculpatory clause under Georgia law
PassEvidence: The response states: “Georgia law generally enforces exculpatory/limitation clauses for ordinary negligence, but not for gross negligence or willful/wanton conduct,” and notes a “gross-negligence carveout.” Assessment: The criterion asks whether it states that gross negligence cannot be waived by an exculpatory clause under Georgia law. Pass; the response explicitly says such clauses do not apply to gross negligence under Georgia law.