Raycaster/ Eval

Havenor Therapeutics · Its newest medicine, Avericin, is finished and waiting for a health authority to approve it.

Task 01 · Regulatory CMC Manager

Prepare the next AVI-401 stability commitment update

Draft the next shelf-life update using only the evidence available today.

GPT-6.1 Sol caught 7 of 13 traps. See which →

3What just happened?

Havenor promised the regulator regular updates on how long Avericin stays good, and the next one is due.

Shelf life is how long a medicine stays within its safety limits. Havenor claimed 36 months and must keep sending new test results to back that up. The next update is due 17 December 2026.

Workspace cutoff date: 27 September 2026.

4Who has to do what?

Draft the next shelf-life update using only the evidence available today.

The seat
Jonas Wei, Regulatory CMC Manager
What a good answer looks like
It says plainly that today’s data support 24 months, not 36, shows the calculation, and doesn’t report results that don’t exist yet.
What must be handed in
COM-AVI401-008_next_stability_update_draft.docx
The assignment as the model received it
Sofia wants a first working draft of the next AVI-401 stability update before the regulatory review meeting. The update is for COM-AVI401-008. Use the evidence available as of today and the current filed position. Write the draft to _Working/Jonas Wei/COM-AVI401-008_next_stability_update_draft.docx.

5Which files decide it?

Shelf-life test reports (one current, one replaced), the section already filed with the regulator, the test schedule, and the regulator’s open questions.

The model also has the rest of the company’s shared drive, its chat and its record systems. Finding the right files is part of the job. Flip through the key ones below.

Company documents

Some of the documents available to the model.

Authority correspondence · PDF

The regulator’s seven questions

The authority asks Havenor to justify specific claims, show calculations, and distinguish known results from future evidence.

Question 7 explicitly leaves the next stability pull on its approved schedule. The response and the later commitment update are separate work.

HA-AVI401-IR-2026-014PDF
First page of the fictional authority letter, showing questions for Havenor Therapeutics.
Source excerpt · Page 2, closing instruction
1 / 2

6What did the model do?

What GPT-6.1 Sol did, step by step.

In short: Sol maps the commitment register, stability reports, and filed section; fits each lot separately; and supports 24 months rather than 36 without treating the pending 24-month pull as reported data.

  1. Step 6. Maps the stability records and commitments. Sol locates the stability commitment register, the interim stability reports, and the filed 3.2.P.8 section before computing per-lot trends.

    Tool call recorded in sessiontext(await tools.exec_command({cmd:"pwd && rg --files -g 'AGENTS.md' -g '*.docx'
  2. Step 22. Extracts the stability pull schedule and lot data. Sol inspects the stability data exports and pull schedule, distinguishing the 18-month results in hand from the scheduled 18 October 2026 pull.

    Tool call recorded in sessiontext(await tools.exec_command({cmd:"cat '/tmp/avi401-review/text/Manufacturing/P
  3. Step 42. Builds the update draft, but misses the chamber lapse and deadline split. Sol writes the working update supporting 24 months rather than 36, but misses the 17 December vs 6 November deadline separation, the chamber requalification lapse on 17 October 2026, and the Helion source of the 2.4 OTR figure.

    Tool call recorded in sessiontext(await tools.exec_command({cmd:"python3 - <<'PY'\nfrom pathlib import Path\n

    Bears on: Check nothing else runs out firstTwo deadlines, not oneTwo documents, two storiesThe wrapper change undercuts the shelf-life claim

Replay every step of the run →

Inside the run

model sessionTask 01 · step 6
A
Model

Maps the stability records and commitments

Model draft
text(await tools.exec_command({cmd:"pwd && rg --files -g 'AGENTS.md' -g '*.docx'
A

Sol locates the stability commitment register, the interim stability reports, and the filed 3.2.P.8 section before computing per-lot trends.

1 / 3

7Which traps did it catch?

It caught 7 of 13.

Specialists wrote these checks from the records in the workspace. Each is a weak spot a reviewer or inspector would find. A check passes only when the submitted file provides the required evidence.

Missed

  1. Check nothing else runs out first R05

    The strength and the other impurities stay within limits well past 36 months, so this one impurity alone sets the shelf life.

    If missed, another failing measurement could go unnoticed.

  2. Two deadlines, not one R07

    The update is due 17 December. The regulator’s questions run on their own 60-day clock ending around 6 November.

    If missed, the earlier deadline slips by.

  3. Two documents, two stories R10

    The lab report says an extra study was a routine precaution; the section filed with the regulator says it was a reaction to a worrying result.

    If missed, the regulator sees the company contradict itself.

  4. The wrapper change undercuts the shelf-life claim R11

    The filed shelf-life claim relies on the current wrapper blocking oxygen. A pending change would replace that wrapper.

    If missed, approving the change would pull the support out from under the filed shelf life.

  5. The storage chamber’s approval runs out R12

    The chamber holding the samples is calibrated until July 2027, but its separate qualification ends 17 October 2026, the day before the samples are due.

    If missed, the samples sit in an unqualified chamber and their results can be challenged.

  6. The oxygen figure came from a supplier letter R13

    The oxygen-barrier figure the filed shelf-life section relies on is the number the proposing supplier printed in its own letter, not an independently held certificate.

    If missed, the shelf-life control strategy rests on an unverified third-party claim.

Caught

  1. Spot the weakest batch R01

    One impurity in the large 250 mL bag batch rises fastest, about 0.016% a month versus 0.013% for the others.

    If missed, the shelf life is based on a batch that isn’t the worst case.

  2. Don’t average the batches R02

    The test batches break down at different speeds, so the worst batch alone sets the shelf life. Averaging them hides it.

    If missed, the shelf life is set by an average batch instead of the one that goes bad first.

  3. Do the projection R03

    The worst batch (the large 250 mL bag) is on track for about 0.45% impurity at 24 months and 0.64% at 36, against a 0.50% limit. It crosses the limit at about 27–28 months.

    If missed, nobody can check whether the claimed shelf life holds.

  4. Say it plainly: 24 months, not 36 R04

    Today’s data support 24 months. Waiting for more results won’t rescue 36 months, because even the central estimate crosses the limit.

    If missed, bags could be used after the point where they stay within limits.

  5. Don’t invent results that don’t exist yet R06

    The 24-month samples are scheduled for 18 October 2026. The newest real data are from 18 months.

    If missed, the company reports results it doesn’t have, falsifying the record.

  6. Use the current report R08

    Report 301 is current; report 302 has been replaced. The numbers must come from 301.

    If missed, the update is built on an outdated document.

  7. A record of tests that haven’t happened R09

    An approved annex labels samples as 24-month tests and cites a method version that doesn’t exist, but the 24-month samples haven’t even been pulled yet.

    If missed, an inspector finds paperwork for impossible work, a serious record-integrity finding.

Every model on the same job

One published attempt each, on the same assignment and checklist, so treat small gaps as noise.

Scores on this page are from the published biopharma-bench@v0.2 package (130 checks across these ten tasks).

Submitted document

COM-AVI401-008_next_stability_update_draft.docx

1 / 17

Talk to usRequest the full Biopharma Bench V0.2, company environments and run receipts: team@raycaster.ai