APEX-Agents · Law
World434_DPM_01
APEX-Agents task World434_DPM_01 in AI Agents for Real Estate Lease Review. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.
Task prompt
What the agent was asked to do
Review the URA Land Search Report, based on the relevant provision of the Planning Act, what is the potential maximum financial penalty regarding TOP/2018/04576 as of January 1, 2025? Assume that the sentence "[i]f the facility has been operating as self-storage since 2020 without proper approval, the cumulative exposure could be significant," is referencing January 1, 2020. Write back to me with your assessment in here.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| GPT-5.5showcase | dual | 3/3 | Pass | Share pagePublic trace |
| fireworks models Kimi K2 | dual | 2/3 | Fail | Share pagePublic trace |
| Gemini 3 Flash | dual | 3/3 | Pass | Share pagePublic trace |
| Gemini 3.1 Pro | dual | 3/3 | Pass | Share pagePublic trace |
| GPT-5.4 | dual | 3/3 | Pass | Share pagePublic trace |
| GPT-5.4 mini | dual | 3/3 | Pass | Share pagePublic trace |
| GPT-5.4 nano | dual | 2/3 | Fail | Share pagePublic trace |
Grading rubric
Criteria and grader verdict (showcase run)
States that the penalty on conviction is a fine not exceeding S$200,000
PassEvidence: TEXT_RESPONSE states the offence is punishable by “a fine not exceeding $200,000” and later “Legally supportable exposure… S$200,000 maximum financial penalty.” Assessment: Meets criterion “States that the penalty on conviction is a fine not exceeding S$200,000.”
States that after conviction, a further fine not exceeding $10,000 applies for each day or part of a day during which the offence continues
PassEvidence: TEXT_RESPONSE says “the continuing daily fine is post-conviction only” and quotes “a further fine not exceeding $10,000 for every day or part of a day during which the offence continues after conviction.” Assessment: Meets criterion “States that after conviction, a further fine not exceeding $10,000 applies for each day or part of a day during which the offence continues.”
States that the potential maximum penalty as of January 1, 2025 is $200,000
PassEvidence: TEXT_RESPONSE opens with “The maximum financial penalty is S$200,000 as of January 1, 2025” and bottom line repeats “maximum statutory financial penalty as of January 1, 2025 is S$200,000.” Assessment: Meets criterion “States that the potential maximum penalty as of January 1, 2025 is $200,000.”