APEX-Agents · Law
World423_AW_01
APEX-Agents task World423_AW_01 in AI Agents for Privacy and GDPR Compliance. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.
Task prompt
What the agent was asked to do
Our client needs to know if any of that data that was transmitted in the breach is considered personal data under GDPR. Review the four attached incident reports and provide your reply to me with exactly the following: 1) a single sentence conclusion identifying if there are any discrepancies among the documents in relation to the data breach; and 2) 1-2 sentences of analysis as to whether the type of data involved in the breach is considered "personal data" under GDPR.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| GPT-5.5showcase | dual | 2/3 | Fail | Share pagePublic trace |
| fireworks models Kimi K2 | dual | 2/3 | Fail | Share pagePublic trace |
| Gemini 3.1 Pro | dual | 2/3 | Fail | Share pagePublic trace |
| GPT-5.4 | dual | 2/3 | Fail | Share pagePublic trace |
| GPT-5.4 mini | dual | 2/3 | Fail | Share pagePublic trace |
| GPT-5.4 nano | dual | 1/3 | Fail | Share pagePublic trace |
Grading rubric
Criteria and grader verdict (showcase run)
States that there is a discrepancy among the documents in relation to the data breach
PassEvidence: TEXT_RESPONSE states, “Yes—there is a discrepancy in the legal/regulatory characterization of the breach...” Assessment: The criterion asks whether the response states there is a discrepancy among the documents in relation to the data breach; pass because it clearly identifies a discrepancy.
States that the Privileged Assessment identified the instance_region descriptor as a pseudonymous identifier
FailEvidence: TEXT_RESPONSE says “the privileged assessment says there was likely no Article 33 reportable breach/no personal data,” and separately says “the RCA ... characterizes the data as pseudonymous event metadata.” Assessment: The criterion asks for stating that the Privileged Assessment identified the instance_region descriptor as a pseudonymous identifier; fail because the response does not mention “instance_region” and attributes pseudonymous characterization to the RCA rather than the Privileged Assessment.
States that a pseudonymous identifier is considered personal data under the GDPR
PassEvidence: TEXT_RESPONSE states, “Under GDPR, the transmitted fields should be treated as personal data if they can be linked, directly or indirectly, to identifiable users,” after referring to “pseudonymous event metadata.” Assessment: The criterion asks whether it states that a pseudonymous identifier is considered personal data under GDPR; pass because the response conveys that linkable/pseudonymous metadata should be treated as personal data under GDPR, albeit with appropriate qualification.