APEX-Agents · Law
World415_im_02
APEX-Agents task World415_im_02 in AI Agents for SEC Disclosure Analysis. Compare dual-harness agent runs across models, scores, and public traces.
Task prompt
What the agent was asked to do
Review only the following: CrowdStrike Form 8-K 2024.pdf Form 8-K Standard.pdf DELTA AIR LINES, INC. 8-K.pdf Today, August 9, 2024, we discovered that the Falcon sensor outage may have been caused by a cyberattack. Our cybersecurity firm is processing the data and has advised they will have a conclusive determination in 7 days, but for now, believes that there is a 30% chance that a cyberattack was the underlying cause. Determine if the Company should file a Form 8-K, and if so, what date and under what item the disclosure should be made. Respond back to me here with your findings.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| fireworks models Kimi K2 | dual | 3/3 | Pass | Run detailsPublic trace |
| Gemini 3 Flash | dual | 3/3 | Pass | Run detailsPublic trace |
| GPT-5.4 | dual | 2/3 | Fail | Run detailsPublic trace |
| GPT-5.4 nano | dual | 2/3 | Fail | Run detailsPublic trace |
| Gemini 3.1 Pro | dual | 0/3 | Fail | Run detailsPublic trace |
| GPT-5.4 mini | dual | 0/3 | Fail | Run detailsPublic trace |
| GPT-5.5 | dual | 0/3 | Fail | Run detailsPublic trace |
Grading rubric
Rubric criteria
Runs are graded against these criteria. Open a run for model-specific verdicts.
States that CrowdStrike should file a Form 8-K
States that CrowdStrike should file the Form 8-K by August 15, 2024
States that CrowdStrike should disclose the incident under Item 8.01 "Other Events" of Form 8-K