Raycaster / evalsAll benchmarks

Cross-benchmark category

Commercial Contract Risk

Compare AI agent runs for Commercial Contract Risk: prompts, scores, grades, and public traces. This page showcases APEX-Agents tasks that test whether AI agents can analyze commercial contract risk, including force majeure provisions, price increase rights, financing controls, and deal-specific risk allocation.

Legal risk analysisForce majeure, price increases, financing control rights
55Tasks
3Primary
1Benchmarks

Published model evidence

A quick view of shareable runs in this category. Open task and trace pages for the prompt, response, files, and grading details.

Gemini 3 Flash56% pass

25/45 runs · avg score 70%

gpt-5.4-mini40% pass

22/55 runs · avg score 56%

Gemini 3.1 Pro33% pass

18/55 runs · avg score 51%

Kimi K231% pass

16/52 runs · avg score 50%

gpt-5.529% pass

16/55 runs · avg score 53%

GPT-5.428% pass

15/54 runs · avg score 50%

Primary tasks

3 tasks with this category as their main focus.

  1. World127_AK_Task03APEX-Agents(task_799bb0f9ca444c7d9c2fff7aaa4885a9)9/10 · Fail

    Based on the client’s SKU data, calculate the weighted average gross margin for each platform. Then determine the percentage price increase required for SKUs on the lowest-margin platform to raise the…

  2. World130_Al-Zhoheir_Hajim_5APEX-Agents(task_5df3da6f8f5c459484670772c82f6941)3/10 · Fail

    Identify the top five technology investments from the Aptean report with the largest positive difference in percentage revenue growth between users and non-users. Include only investments that the rep…

  3. World227_SK_Task09APEX-Agents(task_367f5673fd4641c4b09fd7a42a90e050)0/1 · Fail

    Calculate a normalized market capitalization for AES using data from the comps file and the following approach: - Calculate the volume‑weighted average price (VWAP) for AES Corporation over the 250 t…

Related tasks

52 tasks that also exercise this work as part of a broader assignment.

  1. Analyze Effect of Anti-SLAPP Suit Against BlueLNG on Pre-Suit DiscoveryAPEX-Agents(task_4d143fb2608745848a8f1096667996be)2/4 · Fail

    As counsel to BlueLNG and recently subpoenaed a news reporter under Rule 202 to find out whether he's illegally obtained trade secrets from one of our shipyards. The shipyard filed a motion to dismiss…

  2. Determine Outcome of Shipyard Fire and DelayAPEX-Agents(task_8ec48c4dfa5e4f06b8bac76409c74d83)3/6 · Fail

    Blue Anchor recently sued us (LNG Shipping Inc.) for claims of fraudulent inducement. We filed a motion to compel arbitration after Nakamura experienced a catastrophic fire at its shipyard. The motion…

  3. LawWorld417_AS_02APEX-Agents(task_43de2db73b5a4d899660f13394e80085)5/10 · Fail

    As you know, the new artist montage reel is a hit. Unfortunately, one of the artists featured is not a fan of the wardrobe upgrade - Mara Sings sent a takedown notice, and we initially complied, but w…

  4. Reconcile Compensation, Fee, and Cost Allocation Provisions Across All JV DocumentsAPEX-Agents(task_43ab089534f243eba52ee023a0508ac2)1/1 · Pass

    BlueAnchor served BlueLNG with a notice claiming to exercise its put rights pursuant to the BlueLNG Operating Agreement. Can you please review the OA and confirm whether or not BlueAnchor can actually…

  5. Task 5vd1ebb0APEX-Agents(task_b952b8a8034a4affaaca6d3be7a2d8f7)5/5 · Pass

    We've received the attached warranty claims for some of our products. Please review them. Then, edit the existing existing "product purchases" spreadsheet to show the maximum refund amount a customer …

  6. Task acg14a6cAPEX-Agents(task_5c14b248c88842b1a3442152671cbb18)5/6 · Fail

    Can you review Sections 8.4, 9.1, 9.3,10.1,14.7 in our MSA, against the below outage scenario? A faulty update causes endpoint failures. The customer sends a breach notice by email to a support inbox…

  7. Task awys8050APEX-Agents(task_0cf6902f9ef341328e3261f7055918ee)6/8 · Fail

    CrowdStrike's general counsel sent us a complaint filed in U.S. district court by Larry Stone, alleging violations under Sections 10(b) and 20(a) of the Securities Exchange Act of 1934, as well as Rul…

  8. Task iv36a08aAPEX-Agents(task_7fe79f6934f64f159f8a6c27901e3db8)1/7 · Fail

    Analyze whether Counts 1-7 of Delta's Complaint against CrowdStrike fall within the limitation of liability clause in Section 10.1 of CrowdStrike's standard MSA, indicating "Covered" or "Not Covered" …

  9. Task jdgh600bAPEX-Agents(task_2b7ec4e367344236a14da9094c8d1019)0/3 · Fail

    Can you respond to me, giving a one or two sentence answer to each of the following questions: 1) what is the key principle regarding information exchange from the Joint Venture Antitrust Slides? 2) …

  10. Task jhp9ccb4APEX-Agents(task_a7c1daf7049b46ada402eada03ad7865)2/3 · Fail

    A real-time Rapid Response Content (RRC) update is automatically delivered and it causes performance degradation in a customer’s environment. The customer did not stage deployments, had no rollback pl…

  11. Task yhzc9d1aAPEX-Agents(task_107a64a2aeb8439fb41c5e16d25a5326)5/5 · Pass

    CrowdStrike sent the attached list of historic stock transactions (note the document lists transactions for both Class A and B stocks). We need to determine if any of these people would be a part of…

  12. Task ymtecb81APEX-Agents(task_bf18a6d6ff44489daf0b1c80b5d4a3a8)8/9 · Fail

    Can you take a look at the two Master Supply Agreement templates (Master Supply Agreement Template.pdf ("Template1"), Master Supply Agreement 2.pdf ("Template2"))? We’re considering them for Acme (th…

  13. World 415-CD-02APEX-Agents(task_5b434b51e4de4ad398fa8d9f59321b6e)2/2 · Pass

    Review the following: the statement dated July 19, 2024 from George Kurtz, the Form 8-K from Crowdstrike dated July 19, 2024 and Rule 10b-5. Based solely on these documents, identify any statements ma…

  14. World 431 Task_EL_02APEX-Agents(task_758d25eced8c469d944663dd33220df5)4/5 · Fail

    Go ahead and check the terms of the Letter of Intent ("LOI") and the final draft of the lease to see if there are any inconsistencies between the two documents. If there are, please list each inconsis…

  15. World_418_APA_01APEX-Agents(task_8fb97fca07244d6f843a9a9328084b76)1/10 · Fail

    BlueAnchor wants to get the JV Agreement signed today. Can you send me back a message with a list of items that need to be added, changed, or removed to prepare a final execution version? Do not take …

  16. world415_aeu_01APEX-Agents(task_7952b3923473458ab7c415da7be74810)4/10 · Fail

    We want to get ahead of preparing a settlement agreement for the Delta matter. Can you let me know which of Delta’s original causes of actions are no longer live as we head into the pre-trial conferen…

  17. World415_DM_01APEX-Agents(task_3fa64829d31f43348113cc74f457b3ec)3/3 · Pass

    MLT is a CrowdStrike customer severely affected by the outage. MLT filed a lawsuit for negligence seeking to recover damages arising from the outage. Additionally, MLT successfully transferred venue t…

  18. World415_im_01APEX-Agents(task_4723cf79c09048f684a8a870bf96d187)1/6 · Fail

    An independent investigation of Crowdstrike's Channel File 291 outage revealed that the devices affected were mostly Microsoft or Google devices, and no Apple devices were affected at all. Is another …

  19. World415_im_02APEX-Agents(task_729336fef1614d1a91692ac2bb442bd5)0/3 · Fail

    Review only the following: CrowdStrike Form 8-K 2024.pdf Form 8-K Standard.pdf DELTA AIR LINES, INC. 8-K.pdf Today, August 9, 2024, we discovered that the Falcon sensor outage may have been caused by…

  20. world416_BB_01APEX-Agents(task_0b9134a634c14f24a6c256d034a6c130)2/3 · Fail

    TAC and its supplier executed the first draft of the Master Supply Agreement document (Template 1) as of April 1, 2025. President Trump announced additional tariffs on July 20, 2025. The supplier's r…

  21. World416_DM_01APEX-Agents(task_68bcf4d7fc5045b58045dc9f2f23ce5c)0/3 · Fail

    Due to a riot occurring in response to an Executive Order that resulted in the closure of its factories, TAC sought relief from performance under the force majeure section of the Master Supply Agreeme…

  22. World416_JS_01APEX-Agents(task_2b5df2b5a0eb423e998099a9fcd8e262)2/2 · Pass

    TRIDENT AUTO CORPORATION (The "Plaintiff") has filed a Complaint against United States President Donald Trump (the "Defendant") in the United States District Court for the District of Columbia challen…

  23. World416_JS_02APEX-Agents(task_0097f20dd1744647a5a6e76db215a467)2/2 · Pass

    TRIDENT AUTO CORPORATION (The "Plaintiff") has filed a claim challenging a 25% tariff that President Trump (the "Defendant") imposed on goods imported from China under the International Emergencies Ec…

  24. World416_JS_03APEX-Agents(task_58d04f83ddee4c01bf89a4a1ca0b933a)2/2 · Pass

    TRIDENT AUTO CORPORATION (The "Plaintiff") has filed a claim in the United States District Court for the District of Columbia against several federal government agencies (The "Defendants"). The Claim…

  25. World416_TK_02APEX-Agents(task_3a57257634404d0584a5bd7ce7846b89)0/1 · Fail

    TAC has just informed us that they expect a 20% drop in gross margin due to import tariffs. TAC has posted the information on their website as they have done in the past. Write me back a brief message…

  26. World416_TK_04APEX-Agents(task_5eec5f9ff1234067859443ab0a6c402d)1/2 · Fail

    Review the two supply agreement templates, Master Supply Agreement Template 1.docx and Master Supply Agreement Template 4.pdf, along with the attached files (UCC §2-209 and Restatement (Second) of Con…

  27. World418_AH_01APEX-Agents(task_4c76ef32414a490ebdf5a68aac57e6ca)1/1 · Pass

    Review the shipbuilding contract ("Contract") and draft assignment agreement for the Contract (the "Assignment"). The Assignment will be used by LNG SHIPPING INC. ("LNG") to assign the Contract to its…

  28. World418_BA_01APEX-Agents(task_9509d6400e0c4e67a2222316744e7ded)2/10 · Fail

    Does any agreement between us and BlueAnchor create any non-disclosure obligations preventing our owner from publicly announcing the deal? Write me a short reply back that I can review and send.

  29. World418_EK_01APEX-Agents(task_08c22c6bd072423284c2da7fb190b8bd)3/5 · Fail

    Does assigning the shipbuilding contract to the JV require any other member consents (under the JV Operating Agreement) that we don't already have? Write out your answer to me here, and give a short …

  30. World418-TK-01APEX-Agents(task_3413464bd2b14df6bd2ee15c36d3abf1)1/2 · Fail

    Can you please assess whether or not the new LNG carries built by Nakamura Heavy Industries can be used in Jones Act Trade? Additionally, BlueLNG JV purchased another LNG vessel and asks whether it ca…

  31. World418-TK-02APEX-Agents(task_3bb0664007a140a981193fb43ba1be7b)1/1 · Pass

    Can you please let me know if the canceling date of the shipbuilding contract is 180 days? Just a quick yes or no here is fine.

  32. World418-Tk-03APEX-Agents(task_f1998108c25b4e448ec6cdce9f40321e)2/2 · Pass

    BlueLNG sued Nakamura in the Southern District of New York based on the shipbuilding contract. Given that Nakamura has no office in the district, can BlueLNG seek Rule B attachment of a US$ wire trans…

  33. World419_AH_03APEX-Agents(task_c24da23b0e1042238997125c77506557)0/2 · Fail

    On November 20, 2024, the M/V Red Room struck a submerged object on the bed of the Ohio River. The Incident resulted in a hull breach and the discharge of crude oil into the Ohio River. During subse…

  34. World431_amk_01APEX-Agents(task_c7fc2009b5b849b4b769f0eafe64f461)0/2 · Fail

    MGR Real Estate Inc. (the "Lessor") and "AI Automation Group, LLC" (the "Lessee") entered into the final lease agreement on December 5, 2025 for 2020 Main Street, Irvine, CA (the "Premises"). On De…

  35. World431_amk_02APEX-Agents(task_76585a8b9eb3423895795c8715f924c7)2/5 · Fail

    MGR Real Estate Inc. (the "Lessor") and "AI Automation Group, LLC" (the "Lessee") entered into the final lease agreement on December 5, 2025 (the “Lease”) for 2020 Main Street, Irvine, CA (the "Premis…

  36. World431_AVK_01APEX-Agents(task_611b910ce2514004993effe836ef3d74)2/6 · Fail

    On December 10, 2025 prior to closing on the lease with MGR Real Estate, Inc. ("MGR"), Grace Joblin passed away. The draft consent was not signed by the members of AI Automation Group, LLC ("AIAG") au…

  37. World431_AVK_02APEX-Agents(task_829dd434f17b415c951bd7b7f543d35e)1/1 · Pass

    In addition to the 15,000 sq ft rented to AI Automation Group, LLC (“AIAG”), MGR Real Estate, Inc. ("MGR") rents a total of 157,235 sq ft. of space to the other tenants in the building. Using the FINA…

  38. World431_AVK_03APEX-Agents(task_ccd303088b344ea8b81c13dc04309142)0/2 · Fail

    Review the attached documents to determine the amounts to be distributed from AI Automation Group, LLC ("AIAG") to Shohei Yamamoto and Janet Swift. Get me your reply right back here.

  39. World431_AVK_04APEX-Agents(task_0b6e147c84754379a4e8f3a9057336f8)5/6 · Fail

    Is Agility Tech Solutions LLC's notice to extend the Term of the Lease valid? Please return to me a short explanation based on the lease provisions in your answer, thanks!

  40. World431_DM_01APEX-Agents(task_286b3c8f3efc44299eaa62dd70054bad)3/3 · Pass

    MGR Real Estate Inc. ("MGR") and "AI Automation Group, LLC" ("AIAG") entered into the final lease agreement on December 12, 2025 for 2020 Main Street, Irvine, CA (the "Building"). On December 15, 2025…

  41. World431_DM_02APEX-Agents(task_f31aa2ecb08b4843ab62cba393599947)0/3 · Fail

    MGR Real Estate Inc. ("MGR") and AI Automation Group, LLC ("AIAG") entered into the final lease agreement on December 12, 2025 for 2020 Main Street, Irvine, CA (the "Building"). A couple years later, …

  42. World431_DM_03APEX-Agents(task_65f3283e76254376bb94f65cc9f14897)4/5 · Fail

    Given the information below, is AIAG correct? Print your reply to me here. Give me a Yes/No, and a 1-2 sentence explanation. MGR Real Estate Inc. ("MGR") and "AI Automation Group, LLC" ("AIAG") enter…

  43. World431_DM_04APEX-Agents(task_cba85c18cb7c456da8497c1018c75b16)0/3 · Fail

    MGR Real Estate Inc. ("MGR") and "AI Automation Group, LLC" ("AIAG") entered into a letter of intent (“LOI”) on December 2, 2025 for 2020 Main Street, Irvine, CA (the "Building"). The letter included …

  44. World431_DM_05APEX-Agents(task_18263dbcc46a4fabaf9b7b6f243aa748)3/3 · Pass

    MGR Real Estate Inc. ("MGR") and "AI Automation Group, LLC" ("AIAG") entered into the final lease agreement on December 12, 2025 for 2020 Main Street, Irvine, CA (the "Building"). The Parties mutually…

  45. World431_DM_06APEX-Agents(task_70328d2e54c04d4cb829435037f3c548)0/4 · Fail

    MGR Real Estate Inc. ("MGR") and "AI Automation Group, LLC" ("AIAG") entered into the final lease agreement (the "Lease") on December 12, 2025 for 2020 Main Street, Irvine, CA (the "Building"). On Dec…

  46. World431_jcf_01APEX-Agents(task_55636a0feecb473bbc8d6a447261aec4)0/2 · Fail

    Refer solely to "Lease v3 2020 Main AI Auto CBA 12042025.docx", the version of the lease that was executed. We need a determination on the Commencement Date for the lease. Reply to me right here with …

  47. World431_JS_01APEX-Agents(task_7a420a91d97b4bad8c88e994f8df7768)2/2 · Pass

    2020 Main Street, Irvine (the “Premises”) needs a new HVAC System in order to accommodate INTELLIGENCE INC.'s server rooms. INTELLIGENCE INC. is a tenant at the Premises. The HVAC system will replac…

  48. World431_JS_02APEX-Agents(task_0d1c4041b5e945bc867fa627e302b503)2/2 · Pass

    MGR Real Estate Inc. (The "Lessor") and "AI Automation Group, LLC" (the "Lessee") entered into the final lease agreement on December 5, 2025 for 2020 Main Street, Irvine, CA (the "Premises"). On Decem…

  49. World431_JS_03APEX-Agents(task_d7d932117f2141eba5b8b142fa862a6b)0/2 · Fail

    AI AUTOMATION GROUP, LLC (The "Lessee") is leasing a portion of 2020 MAIN STREET, IRVINE, CA (The "Premises") from MGR REAL ESTATE INC. (The "Lessor"). As part of its business, the Lessee makes intell…

  50. World431_JS_04APEX-Agents(task_78b12b856287427d86e75dadc6ffa465)0/5 · Fail

    MGR REAL ESTATE INC. (The "Lessor") is leasing 2020 MAIN STREET, IRVINE, CA (The "Premises") to AI AUTOMATION GROUP, LLC (The "Lessee"). The Lessee asked their lawyers to identify provisions that are …

  51. World431_JS_05APEX-Agents(task_35d3f7c310b7451fa7400ebc3a0ff372)0/2 · Fail

    MGR REAL ESTATE INC. (The "Lessor") has leased a portion of 2020 MAIN STREET, IRVINE, CA (The "Premises") to AI AUTOMATION GROUP, LLC (The "Lessee"). The lease started on December 5, 2025. The Lesse…

  52. World431Task_EL_01APEX-Agents(task_323b4e29af5e4c7f86eaf4a24c4487ad)4/6 · Fail

    After execution, the lender's lawyers identified issues with the final estoppel certificate. Draft a new estoppel certificate that fixes the issues with the current "final" one. Create a new document…