Raycaster / evalsAll benchmarks

Cross-benchmark category

Energy Grid Resilience

Compare AI agent runs for Energy Grid Resilience: prompts, scores, grades, and public traces. This page showcases APEX-Agents tasks that test whether AI agents can reason about energy grid resilience, curtailment, redispatch, VOLL, and transmission asset risk.

Energy consulting AICurtailment, redispatch, VOLL, transmission asset risk
33Tasks
0Primary
1Benchmarks

Published model evidence

A quick view of shareable runs in this category. Open task and trace pages for the prompt, response, files, and grading details.

Gemini 3 Flash62% pass

8/13 runs · avg score 78%

GPT-5.439% pass

13/33 runs · avg score 58%

gpt-5.4-nano36% pass

12/33 runs · avg score 62%

gpt-5.4-mini36% pass

12/33 runs · avg score 57%

Kimi K233% pass

5/15 runs · avg score 57%

gpt-5.533% pass

11/33 runs · avg score 55%

Related tasks

33 tasks that also exercise this work as part of a broader assignment.

  1. Law_World_423_DM_05APEX-Agents(task_58e8668c0a7c47808ff26e7b1e5105ae)2/9 · Fail

    Northstar's US customer, Zellwerk, reported a system-wide outage that delayed access to shipment-tracking data containing health-related product identifiers and customer account IDs. During the outage…

  2. Task 5vd1ebb0APEX-Agents(task_b952b8a8034a4affaaca6d3be7a2d8f7)5/5 · Pass

    We've received the attached warranty claims for some of our products. Please review them. Then, edit the existing existing "product purchases" spreadsheet to show the maximum refund amount a customer …

  3. Task acg14a6cAPEX-Agents(task_5c14b248c88842b1a3442152671cbb18)5/6 · Fail

    Can you review Sections 8.4, 9.1, 9.3,10.1,14.7 in our MSA, against the below outage scenario? A faulty update causes endpoint failures. The customer sends a breach notice by email to a support inbox…

  4. Task awys8050APEX-Agents(task_0cf6902f9ef341328e3261f7055918ee)6/8 · Fail

    CrowdStrike's general counsel sent us a complaint filed in U.S. district court by Larry Stone, alleging violations under Sections 10(b) and 20(a) of the Securities Exchange Act of 1934, as well as Rul…

  5. Task iv36a08aAPEX-Agents(task_7fe79f6934f64f159f8a6c27901e3db8)1/7 · Fail

    Analyze whether Counts 1-7 of Delta's Complaint against CrowdStrike fall within the limitation of liability clause in Section 10.1 of CrowdStrike's standard MSA, indicating "Covered" or "Not Covered" …

  6. Task jhp9ccb4APEX-Agents(task_a7c1daf7049b46ada402eada03ad7865)2/3 · Fail

    A real-time Rapid Response Content (RRC) update is automatically delivered and it causes performance degradation in a customer’s environment. The customer did not stage deployments, had no rollback pl…

  7. Task Seed #13APEX-Agents(task_ada7c5e9d1a149e780af8d519a9171f6)1/1 · Pass

    During the first 48 minutes of the EU production outage, Northstar's engineering team exported one or two bundled sets of EU production event logs containing personal data to the U.S. analytics vendor…

  8. Task yhzc9d1aAPEX-Agents(task_107a64a2aeb8439fb41c5e16d25a5326)5/5 · Pass

    CrowdStrike sent the attached list of historic stock transactions (note the document lists transactions for both Class A and B stocks). We need to determine if any of these people would be a part of…

  9. World 127_AH_Task 1APEX-Agents(task_e2b897ff18fe4f16a0c969cf70ad2766)0/10 · Fail

    Update numbers with the new projections (attached). I want the full breakdown for: DC converters and onboard chargers Driveline and axle modules Engine control units Engine core hardware Exhaust and …

  10. World 131_MK_Task 1APEX-Agents(task_ec97a505112645f9b266df1954f2738d)0/2 · Fail

    Assuming Eurogrid goes through with the labor reallocation efforts described in the operational efficiency analysis, calculate both the % of total staff that is a manager and the average span of contr…

  11. World 131_MK_Task 2APEX-Agents(task_4b3c2dfc4d164a25831e8787397766c3)4/4 · Pass

    Identify the region with the highest average asset-level Total Score (defined as the sum of Criticality Score, Renewable Impact, and Risk Score), and the country within that region that has the highes…

  12. World 131_MK_Task 3APEX-Agents(task_ee6e8c8f7075427ba5381c453a14c71c)0/5 · Fail

    Please use EuroGrid's headcount per department and the attached benchmarks to calculate the estimated total cost of each of the departments' headcount. Round all final amounts to full USD. Provide yo…

  13. World 131_MK_Task 4APEX-Agents(task_1f6c5b8814fa40bdb25ae11bbd48b6ea)0/4 · Fail

    Please calculate how much Germany's and Netherlands' renewables pipelines (will be only 95%) will cover out of their total yearly loads in 2027 and 2028 in % terms. You can use their average historica…

  14. World 415-CD-02APEX-Agents(task_5b434b51e4de4ad398fa8d9f59321b6e)2/2 · Pass

    Review the following: the statement dated July 19, 2024 from George Kurtz, the Form 8-K from Crowdstrike dated July 19, 2024 and Rule 10b-5. Based solely on these documents, identify any statements ma…

  15. World_131_JR_1APEX-Agents(task_1dba146a5da645b3a27ea3212c0bffc4)4/6 · Fail

    Can you please evaluate the outage causes that affect each country the most, in terms of total outage duration? Categorize hazards relating to flooding or storms as the "Weather - Storm" cause and tho…

  16. World_421_ANB_03APEX-Agents(task_2f4081fb460f4ce79a091be3ffb64b93)7/9 · Fail

    Senior Living Lending, Inc. ("SLL") emailed me because they are concerned that responses to their ad campaigns may fall under the Telemarketing Sales Rule (“TSR”). Can you please draft the content for…

  17. World131_acd_task09APEX-Agents(task_c0476484bd64414f87f46f1868cde2f1)0/3 · Fail

    EuroGrid wants to understand whether the root cause of its asset failures can be explained by age, load, and/or frequency of weather events. Identify the 3 manufacturers with the highest total failur…

  18. World131_acd_task10APEX-Agents(task_5b0050f41fc9495fa8c34b69f1184418)9/9 · Pass

    Take a look at our workforce distribution in the country where we've spent the most on OPEX from 2020-24. Knowing we need 2 line technicians per transmission line, 1 substation technician for each sub…

  19. World131_acd_task11APEX-Agents(task_f32cf6dbffac45eea7454b4f3a62eaf9)0/2 · Fail

    Identify the Phase 1 Assets from the 10 Year Roadmap, assuming that SAIFI / SAIDI hotspots can be defined as assets having SAIFI > 1.0 and SAIDI > 60. Ignore the key criteria for substations and the n…

  20. World131_acd_task12APEX-Agents(task_f9f2907c268c44f686496a4934f1fc15)2/3 · Fail

    Looking only at projects in the Connection Queue that have a status of "Approved" or "Connected", calculate the percentage of the Total Forecasted Demand for the years 2026, 2027, and 2028 that could …

  21. World131_DV_02APEX-Agents(task_d55fe268d7f64a74aacfce4fc374ea96)6/6 · Pass

    Can you calculate the annual EU implied revenue for each Eastern European TSO? Use the midpoint of their implied market share ranges and 40 billion euros as the total market size. Using the implied r…

  22. World131_DV_03APEX-Agents(task_a179d38b095f46eba5eff7baf8f7fd87)0/1 · Fail

    Calculate the NPV from the 12-year cash flow on renewable enablement benefits, considering the following assumptions: - The steady-state annual benefits from renewable enablement mentioned in the busi…

  23. World131_DV_05APEX-Agents(task_9d99126a403e41838e1473d33884ed2f)1/2 · Fail

    Can you state the total simple average of the average implementation cost values, across the various technologies? Use the attached implementation cost deck. Also, state how many technologies have a t…

  24. World131_DV_06APEX-Agents(task_806c8178532944a78b78d565e4bc0313)5/5 · Pass

    What is the net aggregate annual benefit (i.e., total annual savings minus total annual opex) of all of the use cases in the digital use case sizing analysis? According to the new transmission technol…

  25. World131_IB_05APEX-Agents(task_699ea5ca3b0243a5852ad33e33043f12)5/6 · Fail

    Find out the ratio between Curtailment_GWh and Redispatch_GWh and for the lowest average ratio of Country-Region pair, report the average Avoided Curtailment (MWh) and causes of curtailment. Represen…

  26. World131_IB_Task 12APEX-Agents(task_0ea0001d6cb34cc6abf9bf6dc4e8e30b)0/2 · Fail

    Using the Digital Twin Input and Additional bus datasets, identify the Bus IDs associated with renewable energy generation. For each of these Bus IDs, calculate the average (in GW) of their three high…

  27. World131_MD_01APEX-Agents(task_55a201bd877a42eeb0e1868fb6a84230)4/4 · Pass

    Tell me whether or not the asset type that has the highest average adjusted failure probability per outage is also responsible for the highest average Value of Lost Load (VOLL) per asset. VOLL is defi…

  28. World131_MD_03APEX-Agents(task_7acc98cded8b49de972ee79b0461107e)3/3 · Pass

    Investigate whether EuroGrid should consider increasing staffing. Determine if the number of working people per impacted asset is correlated with the expected economic impact of unforeseen downtime in…

  29. world415_aeu_01APEX-Agents(task_7952b3923473458ab7c415da7be74810)4/10 · Fail

    We want to get ahead of preparing a settlement agreement for the Delta matter. Can you let me know which of Delta’s original causes of actions are no longer live as we head into the pre-trial conferen…

  30. World415_DM_01APEX-Agents(task_3fa64829d31f43348113cc74f457b3ec)3/3 · Pass

    MLT is a CrowdStrike customer severely affected by the outage. MLT filed a lawsuit for negligence seeking to recover damages arising from the outage. Additionally, MLT successfully transferred venue t…

  31. World415_im_01APEX-Agents(task_4723cf79c09048f684a8a870bf96d187)1/6 · Fail

    An independent investigation of Crowdstrike's Channel File 291 outage revealed that the devices affected were mostly Microsoft or Google devices, and no Apple devices were affected at all. Is another …

  32. World415_im_02APEX-Agents(task_729336fef1614d1a91692ac2bb442bd5)0/3 · Fail

    Review only the following: CrowdStrike Form 8-K 2024.pdf Form 8-K Standard.pdf DELTA AIR LINES, INC. 8-K.pdf Today, August 9, 2024, we discovered that the Falcon sensor outage may have been caused by…

  33. World423_JS_02APEX-Agents(task_7e51ed8994924d8d9f92938fd8cf9fd2)5/9 · Fail

    It has come to our attention that some of the data transferred by the "Diagnostics Analytics Module" related to residents of Colorado. Does Colorado Law require us to notify Colorado residents of this…