APEX-Agents · Management Consulting
CW134 Aditi 03
APEX-Agents task CW134 Aditi 03 in AI Agents for M&A Legal Due Diligence. Compare dual-harness agent runs across models, scores, and public traces.
Task prompt
What the agent was asked to do
To better understand the market dynamics, the client wants to analyze customer engagement with the mobile app features. Use the feature dataset for competitors from 2016-24 to calculate the following using aggregated data across 2016-24: - For each company, determine the 2024 percentage of customers actively using the in-app analytics feature within the mobile app. - Identify all the companies with instances of In-app Analytics Feature Satisfaction Score (out of 10) > Mobile App Satisfaction Score (out of 10). Round all final answers to the nearest 0.01%. Give me the answer right back here as a short message.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| Gemini 3.1 Pro | dual | 3/6 | Fail | Run detailsPublic trace |
| GPT-5.4 | dual | 3/6 | Fail | Run detailsPublic trace |
| GPT-5.4 mini | dual | 3/6 | Fail | Run detailsPublic trace |
| GPT-5.4 nano | dual | 3/6 | Fail | Run detailsPublic trace |
| GPT-5.5 | dual | 2/6 | Fail | Run detailsPublic trace |
Grading rubric
Rubric criteria
Runs are graded against these criteria. Open a run for model-specific verdicts.
States that % of Syncore customers actively using the in-app analytics feature within the mobile app in 2024 is 66.42%
States that % of Velocity customers actively using the in-app analytics feature within the mobile app in 2024 is 19.00%
States that % of UpSkill customers actively using the in-app analytics feature within the mobile app in 2024 is 7.91%
States that % of TrainIQ customers actively using the in-app analytics feature within the mobile app in 2024 is 0%
States that a company with instances of In-app Analytics Feature Satisfaction Score > Mobile App Satisfaction Score is Syncore
States that one company with instances of In-app Analytics Feature Satisfaction Score > Mobile App Satisfaction Score is Velocity