APEX-Agents · Law
Task jhp9ccb4
APEX-Agents task Task jhp9ccb4 in AI Agents for SEC Disclosure Analysis. Compare dual-harness agent runs across models, scores, and public traces.
Task prompt
What the agent was asked to do
A real-time Rapid Response Content (RRC) update is automatically delivered and it causes performance degradation in a customer’s environment. The customer did not stage deployments, had no rollback plan, and used the system in a critical operational setting. Can you review the board memo, along with Crowdstrike's standard and proposed MSA and let me know if CrowdStrike bears the risk for service interruption caused by the RRC? Explain your response and tell me what documents inform your assessment. Write our your findings to me here. Thanks!
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| fireworks models Kimi K2 | dual | 3/3 | Pass | Run detailsPublic trace |
| GPT-5.4 mini | dual | 2/3 | Fail | Run detailsPublic trace |
| GPT-5.5 | dual | 2/3 | Fail | Run detailsPublic trace |
| GPT-5.4 nano | dual | 1/3 | Fail | Run detailsPublic trace |
| Gemini 3.1 Pro | dual | 0/3 | Fail | Run detailsPublic trace |
| GPT-5.4 | dual | 0/3 | Fail | Run detailsPublic trace |
Grading rubric
Rubric criteria
Runs are graded against these criteria. Open a run for model-specific verdicts.
States that CrowdStrike bears the RRC risk under one of the following: "CrowdStrike MSA.pdf"; "CrowdStrike MSA.docx"; and the standard MSA
States that there is no provision in the MSA that explicitly allocates the RRC risk to the customer
States that there is no provision in the MSA that addresses content deployment