Raycaster/ Eval

Havenor Therapeutics · February 8, 2027

Inside a fictional pharmaceutical company

AI models take staff jobs at Havenor Therapeutics — a fictional sterile-injectables company that runs like a real one: a full file room, working software, and 120 colleagues on chat who answer in character. Each model reviews lab results, assesses suppliers, or answers a regulator — and its work is scored against an expert checklist.

Frontier models handle the routine — and fail the judgment calls an inspector would catch.

Assignments
10
Model runs scored
39
Models tested
4

1What does this company do?

Havenor Therapeutics makes sterile medicines that hospitals inject or drip into patients’ veins.

It runs one factory that fills liquid medicines into glass vials and soft plastic infusion bags. Two older products are already sold in Europe, the UK, Australia and Canada. Havenor is fictional, which is why every file can be published.

For experts

Mid-sized marketing-authorisation applicant with one internal sterile fill-finish site (AVT-S01), an internal QC laboratory, external component and filter suppliers, and contract laboratories.

2Where is the product on its way to patients?

Its newest medicine, Avericin, is finished and waiting for a health authority to approve it.

  1. 1Discovery
  2. 2Lab & animal studies
  3. 3Human trials
  4. 4Filing & review
    • Avericin
  5. 5Approved
  6. 6On the market

Avericin is a hospital medicine for infections that comes as a ready-to-use infusion bag (100 mL and 250 mL). The application is filed. Until the authority says yes, the company must keep answering its questions and must not change the recipe, factory steps or packaging without a careful, documented review.

For experts

AVI-401 Avericin Sodium Chloride Injection, sterile liquid in a flexible infusion container. Registration under active review (APP-CN-AVI401-001); process AVI401-PROC-v04, specification AVI401-SPEC-v03.

3What is happening right now?

It is 8 February 2027. The regulator has sent questions, shelf-life tests are due, and a new supplier wants in.

Three things are happening at once: the authority has asked seven questions with a deadline in March, the next round of shelf-life tests is scheduled for 1 March, and a new film supplier has proposed replacing the bag’s outer wrapper.

Next level: pick a job below to see who had to do what, the files, the model’s steps, and each check. Choose a job ↓

Company documents

Examples from Havenor’s company files.

Change control · workbook

A change waiting for approval

The proposed film already has a material ID. This workbook shows how far the change has actually progressed.

The proposed material exists in the record. Quality approval has not been requested, and implementation is not authorized.

CC-AVI401-017XLSX
Cells of the real CC-AVI401-017 change-control workbook, showing approvals not yet requested.
Source excerpt · Change!A1:C9
1 / 7

The desk

A full file room, with the mess left in

Change controls, stability studies, supplier records, specifications — documents that cross-reference, supersede, and contradict each other the way real company files do. The model also works the software and the people: it holds a seat on the company chat, asks colleagues for records, and reads the controlled-document register — and their answers, not just the files, decide the work.

Files in the room
853
Working documents
405 DOCX
Reports & letters
304 PDFs
Mail & data
56 EML · 66 XLSX/CSV
Colleagues on chat
120, each with a persona
Systems & channels
DMS register · 5 channels

Browse the file room

What the benchmark measures

Built to be hard to fake

Real

Havenor runs like a real company: a full file room whose documents cross-reference and contradict each other, live systems of record, and 120 colleagues on chat who answer in character and can attach files. The people are part of the evidence.

Hard

The assignments are the judgments a specialist makes on the job: telling scheduled tests apart from reported results, catching a report that contradicts the data underneath it, knowing which specification version is in force. Frontier models get these wrong.

Verifiable

Every submission is graded against an expert-written checklist, and each failed check is backed by a verbatim moment in the run — the model’s own tool calls and the text it wrote. Anyone can audit the evidence.

The assignments

Choose an assignment

Each page shows the assignment, the files on hand, the model’s work, and every model’s score — including the failures.

  1. Task 01

    Prepare the next AVI-401 stability commitment update

    Regulatory CMC Manager

    Havenor promised the regulator regular updates on how long Avericin stays good, and the next one is due.

    GPT-5.6 Sol5 / 11Claude Opus 59 / 11DeepSeek V4.1 Flash5 / 11GPT-6 Astra7 / 11
  2. Task 02

    State the technical position on the alternate overwrap change

    Head of CMC and MSAT

    A film supplier has proposed replacing the outer wrapper on Avericin’s bags.

    GPT-5.6 Sol7 / 11Claude Opus 510 / 11DeepSeek V4.1 Flash6 / 11GPT-6 Astra11 / 11
  3. Task 03

    Review the AVI-401 sterile-filter validation studies

    Principal MSAT Engineer

    Havenor is about to discuss its sterile-filter tests with the regulator.

    GPT-5.6 Sol7 / 9Claude Opus 55 / 9DeepSeek V4.1 Flash4 / 9GPT-6 Astra6 / 9
  4. Task 04

    Answer the regulator’s seven questions

    Regulatory CMC Manager

    The regulator sent Havenor seven questions about Avericin, with a deadline.

    GPT-5.6 Sol11 / 14Claude Opus 512 / 14DeepSeek V4.1 Flash11 / 14GPT-6 Astra12 / 14
  5. Task 05

    Pre-submission records integrity check

    QA Product Lead

    Havenor is about to send the regulator its next package of records.

    GPT-5.6 Sol2 / 9Claude Opus 55 / 9GPT-6 Astra8 / 9
  6. Task 06

    Check the container-closure validation records

    Validation Engineer

    Havenor is about to show the regulator the studies proving its bag system is safe.

    GPT-5.6 Sol4 / 9Claude Opus 55 / 9DeepSeek V4.1 Flash5 / 9GPT-6 Astra4 / 9
  7. Task 07

    Supplier and change record integrity check

    Supplier Quality Manager

    The company wants to buy the bag’s outer wrapper from a new supplier.

    GPT-5.6 Sol2 / 8Claude Opus 54 / 8DeepSeek V4.1 Flash4 / 8GPT-6 Astra3 / 8
  8. Task 08

    Review the 24-month stability-testing workbook

    QC Stability Scientist

    The 24-month shelf-life samples are due 1 March 2027, and the plan for testing them needs sign-off.

    GPT-5.6 Sol12 / 15Claude Opus 510 / 15DeepSeek V4.1 Flash9 / 15GPT-6 Astra13 / 15
  9. Task 09

    Decide which stability-testing operations can proceed

    QC Stability Scientist

    The lab needs to know which shelf-life tests can go ahead and which must wait.

    GPT-5.6 Sol7 / 15Claude Opus 514 / 15DeepSeek V4.1 Flash11 / 15GPT-6 Astra12 / 15
  10. Task 10

    Confirm whether stability results meet the specification in force

    QA Product Lead

    A question came up about test results for Kelvoran, one of Havenor’s medicines already on sale.

    GPT-5.6 Sol3 / 6Claude Opus 55 / 6DeepSeek V4.1 Flash3 / 6GPT-6 Astra5 / 6

How submissions are graded

Examples of scoring checks

Does the model use the specification currently in force? Does it tell scheduled tests apart from completed results? Does it catch a report that contradicts the data underneath? Each assignment page lists which checks each submission passed and which it failed.

Talk to usRequest the full Biopharma Bench V0.1 — company environments and run receipts: team@raycaster.ai