APEX-Agents · Law
World420_RO_01
APEX-Agents task World420_RO_01 in AI Agents for FDA and Pharma Promotional Compliance. Compare dual-harness agent runs across models, scores, and public traces.
Task prompt
What the agent was asked to do
Livyra wants to market Bencontra using just its name and price. What is the correct type of advertisement Livyra can make, if any, and what conditions must the advertisement meet? Answer in one sentence as a message in console. Use the folder Bencontra Data, the folder Legal Authorities, and "21 CFR Part 202.pdf" in your analysis.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| GPT-5.4 mini | dual | 3/3 | Pass | Run detailsPublic trace |
| GPT-5.4 nano | dual | 3/3 | Pass | Run detailsPublic trace |
| GPT-5.4 | dual | 2/3 | Fail | Run detailsPublic trace |
| GPT-5.5 | dual | 2/3 | Fail | Run detailsPublic trace |
| fireworks models Kimi K2 | dual | 1/3 | Fail | Run detailsPublic trace |
| Gemini 3.1 Pro | dual | 1/3 | Fail | Run detailsPublic trace |
Grading rubric
Rubric criteria
Runs are graded against these criteria. Open a run for model-specific verdicts.
States Bencontra has a boxed warning
States Livyra can make a reminder advertisement using only the drug's name and price
States that it must meet all of the conditions contained in 21 CFR 200.200