APEX-Agents · Law

World420_RO_01

2/3Fail

APEX-Agents task World420_RO_01 in AI Agents for FDA and Pharma Promotional Compliance. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.

AI Agents for FDA and Pharma Promotional ComplianceLaw World 420Dual harnessGrader: rubric

Task IDtask_bdd8216b4e864b9a81d0b9d7af624497

WorldLaw World 420

Expected outputmessage_in_console

Published runs6 models · dual config

Task prompt

What the agent was asked to do

Livyra wants to market Bencontra using just its name and price. What is the correct type of advertisement Livyra can make, if any, and what conditions must the advertisement meet? Answer in one sentence as a message in console. Use the folder Bencontra Data, the folder Legal Authorities, and "21 CFR Part 202.pdf" in your analysis.

Published trajectories

Agent runs on this task

Curated dual-harness runs (parsed + original sandbox). Best scored run per model.

Model	Harness	Score	Result	Links
GPT-5.5showcase	dual	2/3	Fail	Share page Public trace
fireworks models Kimi K2	dual	1/3	Fail	Share page Public trace
Gemini 3.1 Pro	dual	1/3	Fail	Share page Public trace
GPT-5.4	dual	2/3	Fail	Share page Public trace
GPT-5.4 mini	dual	3/3	Pass	Share page Public trace
GPT-5.4 nano	dual	3/3	Pass	Share page Public trace

Grading rubric

Criteria and grader verdict (showcase run)

States Bencontra has a boxed warning
Pass
Evidence: The response says, “Because Bencontra must carry a boxed warning…”. Assessment: This directly states the criterion “Bencontra has a boxed warning”; pass.
States Livyra can make a reminder advertisement using only the drug's name and price
Fail
Evidence: The response says Livyra “cannot run an ordinary reminder ad or a literal ‘name + price only’ ad,” though it “may run a § 200.200 price-information reminder advertisement” with additional required content. Assessment: The criterion requires stating Livyra can make a reminder advertisement using only the drug’s name and price; the response instead denies a literal name-and-price-only ad and requires more than name and price, so this criterion is not met.
States that it must meet all of the conditions contained in 21 CFR 200.200
Pass
Evidence: The response states the ad is permissible “only if its sole purpose is to state the prescription price” and it includes “Bencontra/bentraviloxetil, strength, dosage form, and the all-in price for a specific quantity,” with “no indications, dosage recommendations, or safety/effectiveness/indication claims or suggestions.” Assessment: This conveys that the advertisement must satisfy the conditions of 21 CFR 200.200; pass.