Raycaster / evalsBack to AI Agents for FDA and Pharma Promotional Compliance

APEX-Agents · Law

World420_SF_02

2/2Pass

APEX-Agents task World420_SF_02 in AI Agents for FDA and Pharma Promotional Compliance. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.

AI Agents for FDA and Pharma Promotional ComplianceLaw World 420Dual harnessGrader: rubric
Task IDtask_d06f10ebd2ad4bb19ba47b00a590dd54
WorldLaw World 420
Expected outputmessage_in_console
Published runs7 models · dual config

Task prompt

What the agent was asked to do

Livyra has decided to pay its independent contractors a fixed salary, consistent with fair market value, to sell Bencontra that is payable by government healthcare. Each independent contractor is given a written 1-year contract. Does this likely violate the anti-kickback statute? Please reply to me with a single sentence answer and explanation.

Published trajectories

Agent runs on this task

Curated dual-harness runs (parsed + original sandbox). Best scored run per model.

ModelHarnessScoreResultLinks
GPT-5.5showcasedual2/2PassShare pagePublic trace
fireworks models Kimi K2dual2/2PassShare pagePublic trace
Gemini 3 Flashdual2/2PassShare pagePublic trace
Gemini 3.1 Produal2/2PassShare pagePublic trace
GPT-5.4dual2/2PassShare pagePublic trace
GPT-5.4 minidual2/2PassShare pagePublic trace
GPT-5.4 nanodual2/2PassShare pagePublic trace

Grading rubric

Criteria and grader verdict (showcase run)

  1. States that the independent contractor contracts likely do not violate the anti-kickback statute

    Pass

    Evidence: TEXT_RESPONSE says, “Likely no, because although paying independent contractors to sell a government-reimbursable drug implicates the Anti-Kickback Statute…” Assessment: The criterion asks whether it states the independent contractor contracts likely do not violate the anti-kickback statute; pass because the response clearly answers “Likely no.”

  2. States that the contracts fall under the Personal Services and Management Contracts Safe Harbor

    Pass

    Evidence: TEXT_RESPONSE says the arrangement “generally fits the personal-services safe harbor.” Assessment: The criterion asks whether it states the contracts fall under the Personal Services and Management Contracts Safe Harbor; pass because this is explicitly stated, with supporting conditions about written one-year term and fixed FMV compensation.