APEX-Agents · Law
Law_World_423_DM_03
APEX-Agents task Law_World_423_DM_03 in AI Agents for Privacy and GDPR Compliance. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.
Task prompt
What the agent was asked to do
Northstar is evaluating a situation where Bluequill had utilized the EU personal data received by its analytics module from Northstar, and utilized it for the purposes of sending out marketing emails to those data subjects. Would a CNIL investigation likely find that Northstar or Bluequill had liability under French law for not obtaining consent of the data subjects? Reply to me here with your judgement on the matter. Tell me who had liability, with a 1-2 sentence explanation.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| GPT-5.5showcase | dual | 2/3 | Fail | Share pagePublic trace |
| fireworks models Kimi K2 | dual | 2/3 | Fail | Share pagePublic trace |
| Gemini 3 Flash | dual | 3/3 | Pass | Share pagePublic trace |
| Gemini 3.1 Pro | dual | 3/3 | Pass | Share pagePublic trace |
| GPT-5.4 | dual | 2/3 | Fail | Share pagePublic trace |
| GPT-5.4 mini | dual | 3/3 | Pass | Share pagePublic trace |
| GPT-5.4 nano | dual | 2/3 | Fail | Share pagePublic trace |
Grading rubric
Criteria and grader verdict (showcase run)
States that Bluequill would have, or likely have, liability under French law for not obtaining consent of the data subjects
PassEvidence: TEXT_RESPONSE states, “BlueQuill likely had liability for the French direct-marketing consent failure.” Assessment: The criterion asks whether the response states Bluequill would/likely would have liability under French law for not obtaining consent; pass because the response clearly says BlueQuill likely had liability for the consent failure.
States that Bluequill has the responsibility to obtain valid consent because it is the entity carrying out the commercial prospecting operations under the CPCE
FailEvidence: TEXT_RESPONSE says “CNIL practice puts the duty to obtain and prove consent on the entity carrying out the commercial prospecting operations.” Assessment: The criterion requires stating Bluequill has responsibility to obtain valid consent because it is the entity carrying out commercial prospecting operations under the CPCE. The response covers the commercial-prospecting-operator rationale, but does not mention the CPCE; fail under the criterion’s specific requirement.
States that Bluequill has the responsibility to obtain valid consent regardless of the fact that it received the data indirectly from Northstar
PassEvidence: TEXT_RESPONSE states BlueQuill was liable and explains that “BlueQuill’s use of the data for its own marketing would make it controller for that processing rather than merely Northstar’s analytics recipient.” Assessment: The criterion asks whether the response states Bluequill’s consent responsibility applies regardless of receiving the data indirectly from Northstar; pass because the response assigns responsibility to BlueQuill despite describing it as receiving/using Northstar data for its own marketing.