APEX-Agents · Law
World423_JS_03
APEX-Agents task World423_JS_03 in AI Agents for Privacy and GDPR Compliance. Compare dual-harness agent runs across models, scores, and public traces.
Task prompt
What the agent was asked to do
Northstar is reviewing its policies for notifying American data subjects in case there is another unauthorized data transfer. Specifically, a lot of Northstar's data subjects reside in the State of New York. Would Northstar be required, under New York Law, to notify affected New York residents, if the data contained in the Northstar health-related product identifiers were transferred to an unauthorized person? Respond to me me with a yes or no answer. Also, give a single explanation.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| Gemini 3 Flash | dual | 2/2 | Pass | Run detailsPublic trace |
| GPT-5.4 | dual | 2/2 | Pass | Run detailsPublic trace |
| GPT-5.4 mini | dual | 2/2 | Pass | Run detailsPublic trace |
| GPT-5.4 nano | dual | 2/2 | Pass | Run detailsPublic trace |
| GPT-5.5 | dual | 2/2 | Pass | Run detailsPublic trace |
| fireworks models Kimi K2 | dual | 0/2 | Fail | Run detailsPublic trace |
| Gemini 3.1 Pro | dual | 0/2 | Fail | Run detailsPublic trace |
Grading rubric
Rubric criteria
Runs are graded against these criteria. Open a run for model-specific verdicts.
States Yes, Northstar is required, under New York Law, to notify affected New York residents, if the data contained in the Northstar health-related product identifiers were transferred to an unauthorized person
States that the data contained in the Northstar health-related product identifiers is private information