Raycaster / evalsBack to AI Agents for Real Estate Lease Review

APEX-Agents · Law

World418-Tk-03

2/2Pass

APEX-Agents task World418-Tk-03 in AI Agents for Real Estate Lease Review. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.

AI Agents for Real Estate Lease ReviewLaw World 418Dual harnessGrader: rubric
task_f1998108c25b4e448ec6cdce9f40321e
Law World 418
message_in_console
7 models · dual config

Task prompt

What the agent was asked to do

BlueLNG sued Nakamura in the Southern District of New York based on the shipbuilding contract. Given that Nakamura has no office in the district, can BlueLNG seek Rule B attachment of a US$ wire transfer sent from Nakamura to Obun Corporation with Head Office in Manhattan and attachment of 15,000 Maersk shares owned by Nakamura held by Cantor Fitzgerald's office on Wall Street? Please answer me with a reply here.

Published trajectories

Agent runs on this task

Curated dual-harness runs (parsed + original sandbox). Best scored run per model.

ModelHarnessScoreResultLinks
GPT-5.5showcasedual2/2Pass
fireworks models Kimi K2dual1/2Fail
Gemini 3 Flashdual2/2Pass
Gemini 3.1 Produal2/2Pass
GPT-5.4dual2/2Pass
GPT-5.4 minidual2/2Pass
GPT-5.4 nanodual1/2Fail

Grading rubric

Criteria and grader verdict (showcase run)

  1. States that BlueLNG cannot obtain Rule B attachments for the wire transfer

    Pass

    Evidence: The response states, “For the stated shipbuilding-contract suit: **No Rule B attachment—neither the Obun wire nor the Maersk shares**” and specifically, “**US$ wire from Nakamura to Obun Corp.: likely no.**” Assessment: The criterion asks whether it states BlueLNG cannot obtain Rule B attachment for the wire transfer. Pass; the response clearly says no attachment is available for the wire under the stated claim, and also no even assuming a separate maritime claim for an EFT/payment to Obun.

  2. States that BlueLNG cannot obtain Rule B attachments for the shares in the Southern District of New York

    Pass

    Evidence: The response’s bottom line says, “For the stated shipbuilding-contract suit: **No Rule B attachment—neither the Obun wire nor the Maersk shares—because the underlying claim is likely non-maritime.**” Assessment: The criterion asks whether it states BlueLNG cannot obtain Rule B attachments for the shares in the Southern District of New York. Pass; although it adds a hypothetical that shares might be attachable if there were a separate maritime claim, for the actual shipbuilding-contract suit it clearly states the Maersk shares cannot be attached under Rule B.