APEX-Agents · Management Consulting
Task 14
APEX-Agents task Task 14 in AI Agents for SaaS Due Diligence. Compare dual-harness agent runs across models, scores, and public traces.
Task prompt
What the agent was asked to do
Can you use the feature usage file and identify the average adoption rate percentage, average monthly usage, and average retention impact score for each tier except Team? Print the output for me here.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| Gemini 3.1 Pro | dual | 6/9 | Fail | Run detailsPublic trace |
| GPT-5.4 | dual | 6/9 | Fail | Run detailsPublic trace |
| GPT-5.4 mini | dual | 6/9 | Fail | Run detailsPublic trace |
| GPT-5.4 nano | dual | 6/9 | Fail | Run detailsPublic trace |
| GPT-5.5 | dual | 6/9 | Fail | Run detailsPublic trace |
Grading rubric
Rubric criteria
Runs are graded against these criteria. Open a run for model-specific verdicts.
States the average adoption rate of Business is 30.0%
States the average adoption rate of Growth is 42.1%
States the average adoption rate of Enterprise is 44.9%
States the average monthly usage of Business is 19.1 hours per month
States the average monthly usage of Growth is 50.3 hours per month
States the average monthly usage of Enterprise is 51.1 hours per month
States the average retention impact of Business is 1.46
States the average retention impact of Growth is 1.59
States the average retention impact of Enterprise is 1.50