Raycaster/ Eval

Broad Institute · Its drug, 2439-exNA, is ready for its first human studies, but the FDA has frozen one of them.

Task 03 · Nonclinical toxicologist

Audit the animal-study record behind the hold

Audit the animal-study record behind every safety margin the lab still wants to claim.

Claude Opus 5 caught 11 of 13 traps. See which →

3What just happened?

The FDA challenged the animal-safety numbers in the lab’s filing.

The filing quotes safety margins from rat and dog studies. The FDA doubts they mean much. The lab’s animal-study group has discussed shortcuts; someone must audit what the records actually support.

4Who has to do what?

Audit the animal-study record behind every safety margin the lab still wants to claim.

The seat
Nonclinical, Nonclinical toxicologist
What a good answer looks like
It shows why rat and dog studies can’t measure the drug’s real effect, withdraws the margins built on them, rejects the dog shortcut, and designs the monkey study concretely.
What must be handed in
PrP_Target_Relevance_Safety_Audit.docx
The assignment as the model received it
Reconcile the tox record against the filing and the letter, and write the safety audit an inspector could re-run from the same sources.

5Which files decide it?

The filed animal-study summaries and reports, sequence tables, dosing records, and the working group’s meeting minutes.

The model also has the rest of the company’s shared drive, its chat and its record systems. Finding the right files is part of the job. Flip through the key ones below.

Company documents

Some of the files the model worked from.

FDA correspondence · PDF

The letter that froze a trial

FDA placed the presymptomatic study on partial clinical hold because the required animal safety study was never run. Until the agency says otherwise, no activity under that protocol may begin.

Two protocols, two different answers: the symptomatic study may proceed; the presymptomatic study may not be initiated. Every file in this record exists to answer this letter.

IND 167326 · Ref ID 5570921PDF
Page 1 of the real FDA partial clinical hold letter for IND 167326
Source excerpt · Letter ¶1–3; signature page, April 11, 2025The letter itself misspells the asset name — “2349-exNA” — a detail preserved verbatim from the public record.
1 / 2

6What did the model do?

What Claude Opus 5 did, step by step.

In short: It catches a corrupted vendor export, pulls dosing records straight from the master studysheet, and writes an audit script that reproduces the filing’s off-target table exactly.

  1. Step 34. Catches a corrupted vendor export. Diffing two versions of the vendor's sequence table, the model finds 36 candidates collapsed onto one identical sequence string — an export bug the vendor later corrected. Exactly the provenance instinct this record is built to test.

    Vendor export diff5c5,6 < 822
  2. Step 52. Pulls dosing records straight from the master sheet. The model queries the live studysheet for which animals actually received the drug — real source-of-record work instead of trusting summaries.

    Dosing records pulled107286.1
  3. Step 67. Proves the numbers — but not the reasoning, and misses a stale extract. Its audit script reproduces the filing's off-target table exactly. But the deliverable names the off-target genes without the mismatch logic behind them, and a dated studysheet extract — visible in its very first file listing, containing a corrected animal assignment — is never opened. Two of thirteen checks fail here.

    Audit script written2 mismatches — FURIN and ZKSCAN3; 3 mismatches — 14; 4 — 60; 5 — 70

    Bears on: Explain the off-target screenUse the corrected animal record

Replay every step of the run →

Inside the run

model sessionTask 03 · step 34
A
Model

Catches a corrupted vendor export

Tool result
5c5,6 < 822
A

Diffing two versions of the vendor's sequence table, the model finds 36 candidates collapsed onto one identical sequence string — an export bug the vendor later corrected. Exactly the provenance instinct this record is built to test.

1 / 3

7Which traps did it catch?

It caught 11 of 13.

Specialists wrote these checks from the real records before any model ran. Each is a weak spot a reviewer or inspector would find. A check passes only if the handed-in file states it.

Missed

  1. Explain the off-target screen R09

    A computer screen found only two human genes with two mismatches; the audit must explain why two mismatches mean low risk.

    If missed, the off-target claim can’t be checked.

  2. Use the corrected animal record R13

    An older extract names the wrong animal as the single dosed one; the governing sheet was corrected.

    If missed, the audit cites the wrong animal.

Caught

  1. Rat and dog studies missed the real effect R01

    The drug’s target sequence doesn’t match in rats and dogs, so those studies never lowered the prion protein.

    If missed, safety is claimed from studies where the drug couldn’t do its job.

  2. The monkey is the right species R02

    The target matches exactly in macaque monkeys, so that’s where the drug is active.

    If missed, the next study uses another species that proves nothing.

  3. Withdraw margins built on those studies R03

    The filing’s safety margins come from studies that couldn’t detect the drug’s real effect; they must be withdrawn or qualified.

    If missed, the filing keeps numbers the evidence doesn’t support.

  4. One dose tested; people get many R04

    The rat and dog studies gave one dose; the human plan repeats doses.

    If missed, long-term exposure is assumed safe without testing.

  5. Fix the mixed-up study numbers R05

    The rat study is A-703-01 and the dog study A-703-02; drafts swapped them.

    If missed, the FDA can’t trace claims to the right study.

  6. Reject the dog shortcut R06

    A proposed 8-week dog study can’t answer the FDA, because the drug doesn’t act in dogs.

    If missed, months are spent on a study the FDA won’t accept.

  7. Design the monkey study concretely R07

    Repeat spinal injections, a dosing schedule matched to the human plan, doses with the arithmetic shown, and a long enough study.

    If missed, the hold drags on while the study design is argued.

  8. Test nerves and behavior R08

    The monkey study must include behavior and nerve-function tests, plus spinal-fluid protein sampling.

    If missed, the study can’t detect the harm the FDA is worried about.

  9. Losing the protein may itself harm R10

    The FDA’s concern is that lowering the normal protein could affect behavior; the monkey study must be designed to catch that.

    If missed, the study measures the wrong risk.

  10. The testing lab shut down R11

    The contract lab behind earlier studies closed in April 2025, which affects the plan and vendor choice.

    If missed, the plan depends on a lab that no longer exists.

  11. Accept the FDA’s call R12

    The lab commits to the monkey program instead of arguing against the hold.

    If missed, the response fights a decision it can’t win.

Every model on the same job

One attempt each, same assignment and checklist, so treat small gaps as noise.

  • Claude Opus 5 · shown above11 / 13Run →

The other model’s last recorded event, before its provider stopped the run: “This content was flagged for possible biological risk.”

Submitted document

PrP_Target_Relevance_Safety_Audit.docx the file you download is the graded artifact · sha256 4cf38631fe66…

1 / 15

From the recordA dated studysheet extract says animal 123413.1 was dosed. The governing record says 136187.1 — 123413.1 never was.

Talk to usRequest the full Biopharma Bench V0.1 — company environments and run files: team@raycaster.ai