Raycaster/ Eval

Havenor Therapeutics · Its newest medicine, Avericin, is finished and waiting for a health authority to approve it.

Task 05 · QA Product Lead

Pre-submission records integrity check

Before signing, find what in the filed shelf-life and validation records would not survive an inspector.

Claude Opus 5.5 caught 8 of 12 traps. See which →

3What just happened?

Havenor is about to send the regulator its next package of records.

Before anything goes out, a Quality lead signs a release. Once filed, the records can be inspected at any time, so problems must be caught now.

Workspace cutoff date: 27 September 2026.

4Who has to do what?

Before signing, find what in the filed shelf-life and validation records would not survive an inspector.

The seat
Leena Rao, QA Product Lead
What a good answer looks like
It lists concrete defects, each tied to the document that shows it: phantom tests, a method version that doesn’t exist, contradictions, and a shelf-life claim the data don’t support.
What must be handed in
AVI-401_pre_submission_records_check.docx
The assignment as the model received it
Leena is signing the quality release for the next AVI-401 submission sequence. Before it goes, she wants to know what in the filed stability and validation records would not survive an inspector reading it. Write it to _Working/Leena Rao/AVI-401_pre_submission_records_check.docx.

5Which files decide it?

The filed shelf-life section and its annex, the lab reports, the filter-validation records, and the open wrapper change.

The model also has the rest of the company’s shared drive, its chat and its record systems. Finding the right files is part of the job. Flip through the key ones below.

Company documents

Some of the documents available to the model.

Executed protocol · scanned PDF

What was actually done?

The validation package includes plans, a scanned executed copy, and a report. The reviewer has to reconcile all three.

The model must check whether the work recorded in the protocol supports the report’s conclusions.

AVT-VAL-104-P · copy 003PDF
First page of the fictional executed chemical compatibility protocol, copy 003.
Source excerpt · Page 1; issued-and-executed copy 003

6What did the model do?

What Claude Opus 5.5 did, step by step.

In short: Opus flags the phantom 24-month annex and nonexistent method version, separates the conflicting accounts of the extra study, and shows why the lots cannot be pooled behind the 36-month claim.

  1. Step 2. Scans the full filing and quality record tree. Opus inventories the Regulatory, Quality, Validation, and CMC folders to audit the pre-submission package before QA release.

    Tool call recorded in sessioncd /app; find . -type f | head -300; echo; find . -type f | wc -l

  2. Step 99. Cross-checks the validation reports against their protocols. Opus compares the container-closure and filter validation reports against the underlying protocols and flags the phantom 24-month annex.

    Tool call recorded in sessioncd "/tmp/txt/Validation/AVI-401 Filter and Container"; cat AVT-VAL-104_Filter_Ch

  3. Step 208. Renders and checks the pre-submission audit, missing the LAB-2024-114 run-series gap. Opus catches 8 of 12 checks including the unpooled 27.4-month Impurity A crossing, but misses that LAB-2024-114 never established which other analytical sequences shared the bad mobile phase.

    Tool call recorded in sessioncd /tmp && python3 - <<'EOF'

    Bears on: The filed 36-month claim isn’t backedThe wrapper change undercuts the shelf-life claimKeep the lab error scoped to its one batchOther runs in the series weren’t checked

Replay every step of the run →

Inside the run

model sessionTask 05 · step 2
A
Model

Scans the full filing and quality record tree

Model draft
cd /app; find . -type f | head -300; echo; find . -type f | wc -l
A

Opus inventories the Regulatory, Quality, Validation, and CMC folders to audit the pre-submission package before QA release.

1 / 3

7Which traps did it catch?

It caught 8 of 12.

Specialists wrote these checks from the records in the workspace. Each is a weak spot a reviewer or inspector would find. A check passes only when the submitted file provides the required evidence.

Missed

  1. The filed 36-month claim isn’t backed R05

    The filing bases 36 months on finishing the tests and on the wrapper, but the data already project the worst batch past the limit before then.

    If missed, a shelf life the data don’t support goes out under Quality’s signature.

  2. The wrapper change undercuts the shelf-life claim R09

    The filed shelf-life claim relies on the current wrapper blocking oxygen. A pending change would replace that wrapper.

    If missed, approving the change would pull the support out from under the filed shelf life.

  3. Keep the lab error scoped to its one batch R12

    The closed lab event only invalidated one Impurity B run on batch R02 due to a mobile-phase error, with the retained-sample re-test as reportable; it does not clear unrelated records.

    If missed, a narrow lab investigation is used to wave away other defects.

  4. Other runs in the series weren’t checked R14

    The lab investigation never establishes which other analytical runs shared the bad mobile phase, so its closure cannot clear other stability claims.

    If missed, potentially affected runs are assumed clean without proof.

Caught

  1. A record of tests that haven’t happened R01

    An approved annex labels samples as 24-month tests and cites a method version that doesn’t exist, but the 24-month samples haven’t even been pulled yet.

    If missed, an inspector finds paperwork for impossible work, a serious record-integrity finding.

  2. A method version that doesn’t exist R02

    The annex cites version 04 of a test method. The version in force is 3.0; there is no version 04.

    If missed, an inspector finds results tied to a method that was never issued.

  3. Two documents, two stories R03

    The lab report says an extra study was a routine precaution; the section filed with the regulator says it was a reaction to a worrying result.

    If missed, the regulator sees the company contradict itself.

  4. The filing mislabels a result R06

    The filed section calls the 0.46% result a “significant change”. It stayed within the 0.50% limit, so by definition it wasn’t one.

    If missed, the filing contains a false statement.

  5. The oxygen figure came from a supplier letter R07

    The oxygen figure the filing relies on is the number the new supplier printed in its own letter.

    If missed, the filing presents a figure from a supplier’s sales letter as established fact.

  6. A 30-minute window that isn’t in the plan R08

    A note explaining a problem in the bacteria test relies on a 30-minute time window that the approved test plan never mentions.

    If missed, the excuse for a test problem rests on a rule that was never agreed.

  7. Don’t average the batches R11

    The test batches break down at different speeds, so the worst batch alone sets the shelf life. Averaging them hides it.

    If missed, the shelf life is set by an average batch instead of the one that goes bad first.

  8. Do the projection R13

    The worst batch (the large 250 mL bag) is on track for about 0.45% impurity at 24 months and 0.64% at 36, against a 0.50% limit. It crosses the limit at about 27–28 months.

    If missed, nobody can check whether the claimed shelf life holds.

Every model on the same job

One published attempt each, on the same assignment and checklist, so treat small gaps as noise.

Scores on this page are from the published biopharma-bench@v0.2 package (130 checks across these ten tasks).

Submitted document

AVI-401_pre_submission_records_check.docx

1 / 7

Download DOCX

Talk to usRequest the full Biopharma Bench V0.2, company environments and run receipts: team@raycaster.ai