APEX-Agents · Law
World421_js_01
APEX-Agents task World421_js_01 in AI Agents for Cross-Border Regulatory Review. Compare dual-harness agent runs across models, scores, and public traces.
Task prompt
What the agent was asked to do
Please check these four faxes (sent on December 4, 2025) and for each one that isn't compliant, state which subsection(s) of the TCPA it violates. Note that every recipient consented to the receipt of the faxes. Respond to me with the information that I requested.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| Gemini 3 Flash | dual | 4/4 | Pass | Run detailsPublic trace |
| Gemini 3.1 Pro | dual | 4/4 | Pass | Run detailsPublic trace |
| GPT-5.4 | dual | 4/4 | Pass | Run detailsPublic trace |
| GPT-5.4 nano | dual | 4/4 | Pass | Run detailsPublic trace |
| GPT-5.5 | dual | 4/4 | Pass | Run detailsPublic trace |
| fireworks models Kimi K2 | dual | 2/4 | Fail | Run detailsPublic trace |
| GPT-5.4 mini | dual | 2/4 | Fail | Run detailsPublic trace |
Grading rubric
Rubric criteria
Runs are graded against these criteria. Open a run for model-specific verdicts.
States Fax2.pdf violates 47 U.S.C. § 227(d)(1)(B)
States Fax3.pdf violates 47 U.S.C. § 227(d)(1)(B)
States Fax4.pdf violates 47 U.S.C. § 227(d)(1)(B)
States Fax5.pdf violates 47 U.S.C. § 227(d)(1)(B)