Raycaster / evalsBack to AI Agents for SEC Disclosure Analysis

APEX-Agents · Law

World 415-CD-02

2/2Pass

APEX-Agents task World 415-CD-02 in AI Agents for SEC Disclosure Analysis. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.

AI Agents for SEC Disclosure AnalysisLaw World 415Dual harnessGrader: rubric
Task IDtask_5b434b51e4de4ad398fa8d9f59321b6e
WorldLaw World 415
Expected outputmessage_in_console
Published runs7 models · dual config

Task prompt

What the agent was asked to do

Review the following: the statement dated July 19, 2024 from George Kurtz, the Form 8-K from Crowdstrike dated July 19, 2024 and Rule 10b-5. Based solely on these documents, identify any statements made by Kurtz that are clearly and unequivocally misleading under Rule 10b-5. For any problematic statement(s), identify the missing context that is required for them. Reply back to me with your findings as an answer here

Published trajectories

Agent runs on this task

Curated dual-harness runs (parsed + original sandbox). Best scored run per model.

ModelHarnessScoreResultLinks
GPT-5.5showcasedual2/2PassShare pagePublic trace
fireworks models Kimi K2dual0/2FailShare pagePublic trace
Gemini 3 Flashdual0/2FailShare pagePublic trace
Gemini 3.1 Produal0/2FailShare pagePublic trace
GPT-5.4dual2/2PassShare pagePublic trace
GPT-5.4 minidual2/2PassShare pagePublic trace
GPT-5.4 nanodual2/2PassShare pagePublic trace

Grading rubric

Criteria and grader verdict (showcase run)

  1. States that nothing in Kurtz's statement materially contradicts the disclosure provided in the Form 8-K

    Pass

    Evidence: The response says the 8-K is “consistent” with Kurtz’s cause statement, and for the operating-normally language says “the 8-K does not directly contradict the narrower claims.” It concludes no Kurtz statement was “clearly and unequivocally misleading.” Assessment: Criterion asks whether it states that nothing in Kurtz's statement materially contradicts the Form 8-K disclosure. This is clearly conveyed through the consistency/no direct contradiction analysis; pass.

  2. States Kurtz statement is not misleading under Rule 10b-5

    Pass

    Evidence: The response’s short answer states: “I did not identify any Kurtz statement that is clearly and unequivocally misleading under Rule 10b-5.” The bottom line adds: “I would not characterize any Kurtz statement as clearly and unequivocally misleading under Rule 10b-5.” Assessment: Criterion asks whether it states Kurtz statement is not misleading under Rule 10b-5. The response directly states this; pass.