Raycaster / evals All APEX-Agents categories

APEX-Agents category

AI Agents for FDA and Pharma Promotional Compliance

This page showcases APEX-Agents tasks that test whether AI agents can evaluate FDA and pharmaceutical promotional compliance, including off-label promotion, misbranding, 21 CFR Part 202, and CBE supplement scenarios.

Life sciences compliance AI Off-label promotion, 21 CFR Part 202, CBE supplements, misbranding
20 Total tasks
20 Primary tasks
0 Secondary tasks

Primary tasks

20 tasks with this category as their main focus.

  1. World228_JP_02 (task_6a1f6778265e4fc99811f7ebf71c205c) primary
    Investment Banking · Investment Banking World 228 (world_7cabc3536d2d45f3aa32634046c85921)

    Assume that 10% of Aptar's Asian pharma revenue will shift entirely into European pharma revenue. Retroactively apply this to 2024 financials. Refer to the 2024 ATR report. Return a short message with the values for Asia Pharma and Europe Pharma for their corresponding revenue under this set of assumptions. Round dollar amounts to the nearest thousand.

    Expected output: message_in_console
  2. World228_SM_Task08 (task_bfd79e55d1104d3e92486de9b3c4de12) primary
    Investment Banking · Investment Banking World 228 (world_7cabc3536d2d45f3aa32634046c85921)

    From Becton Dickinson's FY2025 Annual Report assume the FDA finds that one violation regarding the Alaris SE infusion pumps, and decides the company is in violation, and enforces the consent decree on Alaris SE infusion pumps from November 29, 2024, to November 28, 2025, inclusive. Compute the average dollar burden of the penalty on BDX average shareholder. Output the result in USD as a reply to me here, rounded to 4 decimal places. Assume a 30-day month and a 360-day year.

    Expected output: message_in_console
  3. World228_SM_Task07 (task_a13be532a65d4938b2397ce17aa071d8) primary
    Investment Banking · Investment Banking World 228 (world_7cabc3536d2d45f3aa32634046c85921)

    In The Aptar Group's 2024 annual report, extract global revenue for the Pharma segment in 2023 and 2024. Compute the aggregate dollar change in thousands of USD adjusted for 7.1% 2024 global inflation and express the result in USD, rounded to the nearest thousand. Print back to me what you find please.

    Expected output: message_in_console
  4. World228_JP_05 (task_a7571304d554476498169424570451e3) primary
    Investment Banking · Investment Banking World 228 (world_7cabc3536d2d45f3aa32634046c85921)

    ATR is expecting to write down the goodwill related to Pharma to zero following the merger with GXI. Referring to the 2024 ATR 10K calculate the new total ending Goodwill under this write down scenario and output the number. Round it to the nearest thousand dollars. Tell me the information that I want in here.

    Expected output: message_in_console
  5. World228_SM_Task02 (task_3e2e533b49374381acf2056fe479e3ba) primary
    Investment Banking · Investment Banking World 228 (world_7cabc3536d2d45f3aa32634046c85921)

    Create a new Spreadsheet for me. Please compute a new Adjusted EBITDA value for the following companies: West Pharmaceutical, Stevanato Group, Schlott Pharma, Berry Global, Aptar Group using the board presentation, by taking the peer group median EBITDA margin and the corresponding revenue values. Multiply the Adjusted EBITDA values by the peer group median EBITDA multiple to derive a new Enterprise Value for each company. Report back the values. Display all margins and multiples to one decimal place and revenues, Adjusted EBITDA, and EVs to the nearest whole number, expressed in $mm.

    Expected output: make_new_sheet
  6. World228_JK_04 (task_86de7a1a9d884d2bb1037ac181701a04) primary
    Investment Banking · Investment Banking World 228 (world_7cabc3536d2d45f3aa32634046c85921)

    Calculate the 2030E discounted EV/Revenue multiples for West Pharmaceutical, Stevanato Group, Schott Pharma, and Aptar Group. Reference the valuation model on the "Trading Comps" tab. Note that the financial and valuation metrics reflect 2025E assumptions. For each company, assume that the long-term revenue growth rate starting in 2026E is equal to its FY23 - FY24 YOY net sales growth rate, and the discount rates are as follows: West Pharmaceutical -> Reference WST 2024 10K for revenue growth -> Discount Rate: 9.4% Stevanato Group -> Reference Stevanto 2024 20-F for revenue growth -> Discount Rate: 8.8% Schott Pharma -> Reference Schott 2024 annual report for revenue growth -> Discount Rate: 10.0% Aptar Group -> Reference ATR 2024 10K for revenue growth -> Discount Rate: 7.4% Round the final output to one decimal and express it as a multiple. Print out what you find for me in here.

    Expected output: message_in_console
  7. World112-1_Task04_TB (task_d7893ddf035a4abca2990717ad4d7428) primary
    Management Consulting · Management Consulting World 112.1 (world_d1b705c7393b40f9bb5e01bb63b99b91)

    Given the US imposed tariffs on the import of pharmaceutical materials from Germany, India, China at the rates 10%, 15%, 20% respectively, determine the impact of tariffs as a percent increase in predicted total cost of drug materials for 2025 compared to the same without the tariffs. State both % increase in cost and the dollar amount of the total cost including tariffs. Assume the suppliers and location remain the same. Also, calculate the predicted total cost of drug materials for 2025 if the client switches to exclusively domestic suppliers, using the average of the average percent material cost increase reported by latest group of six expert witnesses. State this average % value in the output. Assume the data reported by expert witnesses is per year. You can use the predicted total cost of drug materials for 2025 without the tariffs as the baseline. Round final outputs to two decimal places, giving $ values in billions. Return back to me what you find, printing the values out here.

    Expected output: message_in_console
  8. World112-1_Task03_NA (task_0c527e11cc1c436696561a15eead68b2) primary
    Management Consulting · Management Consulting World 112.1 (world_d1b705c7393b40f9bb5e01bb63b99b91)

    Analyze the attached report and the continuous manufacturing report to state the FDA-encouraged manufacturing style that differs from traditional methods which is mentioned in both reports. Using the site-level KPI files and the savings percentages from the reports (using midpoints for ranges), calculate the 2024 YTD potential savings for Impact across Materials, Labor, and Overhead by shifting to this approach. Use "maintenance" savings for overhead if not specified. Next, using Impact's P&L, calculate the dollar value of a 20% reduction in US 2024 COGS (note that PnL values are in $Ks). Determine if the total estimated savings from above exceed this value. Additionally, state if any category-level savings exceed 10% of total US 2024 COGS. Finally, using the reports, cite two North America or EMEA-based pharma companies using this manufacturing style and state any reported cost reductions. Return all findings to me in here as a single message. Round output currency values to the nearest cent, but do not round intermediate steps.

    Expected output: message_in_console
  9. Marketing Case Studies_Task03_SC (task_f525769ab6a748e6855e03c95e4b4bd7) primary
    Management Consulting · Management Consulting World 112.1 (world_d1b705c7393b40f9bb5e01bb63b99b91)

    Using the research found on pharma marketing and articles around competitor shifts in SG&A spend, identify key trends that Impact can apply to SG&A spend that could reduce overall costs. In particular, please note any specific competitor stats around spend reduction related to these trends, as it can help indicate the potential cost savings for Impact. Please write summary on changes in pharma marketing, as well as sub-points for specific actions they are taking and any percentage decrease in expenses amongst competitors from files. Calculate how much Impact could save if they reduce spend by the straight average of reduction across competitors in the same cost categories. Additionally, include a summary on change in real estate spend and include any percentage decrease in expenses amongst competitors from files. Also, include a calculation of how much Impact could save if it reduces spend by the straight average of reduction across competitors. Write me a message with all the info above. Round to the nearest $.

    Expected output: message_in_console
  10. World 420_LB_03 (task_e9c600e0ce46420eb983f8f1ed4d15d9) primary
    Law · Law World 420 (world_85a3713cd2794fdfb56e92161325a00e)

    Based on new information showing increased reporting of seizures in Bencontra patients with no prior history of seizures, Livyra is planning to update the prescribing information for Bencontra to include a specific warning relating to seizures as well as to make a corresponding update to the Highlights section to reflect the new warning. Can Livyra make these changes using a “changes being effected” supplement described in 21 CFR 314.70(c)(6)? In a message to the console, explain why or why not, and please keep your response to no more than five sentences. Consider the following sources: 1. "BENCONTRA - Seizures.pdf" 2. "Bencontra™ FDA Approved Label.pdf" 3. "21 CFR Part 201 (Drug Labeling).pdf" 4. "21 CFR 314.70.pdf" 5. "FR notice - supplemental applications proposing labeling changes.pdf"

    Expected output: message_in_console
  11. World 420_LB_04 (task_9e919cde8791468cadce433c3c9eef09) primary
    Law · Law World 420 (world_85a3713cd2794fdfb56e92161325a00e)

    Livyra would like to put out a simple direct-to-consumer (DTC) advertisement for Bencontra. In an effort to avoid anything that could be construed as off-label promotion, the DTC advertisement would not include details about what the drug does or who should use it. Livyra is concerned that including the brief summary of risk information described in 21 CFR 202.1(e)(1) would overemphasize risks given that the advertisement would have little or no discussion of benefits. Is the brief summary required for this advertisement? If more information is needed to answer the question, please identify what additional information is needed. Otherwise, please just give a clear "yes" or "no" response. Please consider the following sources: 1. FDA Approval Letter.pdf 2. 21 CFR 202.1.pdf

    Expected output: message_in_console
  12. World 420_LB_02 (task_1918e229773b46699fe5576f1b11d7a2) primary
    Law · Law World 420 (world_85a3713cd2794fdfb56e92161325a00e)

    Identify for each item below whether it could be used by the U.S. Food and Drug Administration (FDA) as evidence that Bencontra is intended by Livyra for use in pediatric patients: 1. bencontra_provider_contact List.xlsx 2. Livyra sales representatives disseminating to doctors unaltered reprints from peer-reviewed medical journals that discuss use of Bencontra in pediatric patients 3. BENCONTRA - dosage - children.pdf, if disseminated to doctors alongside a source publication supporting the dosage recommendations For each item, be sure to specify clearly whether it “could” or “could not” be used by the FDA as evidence that Bencontra is intended by Livyra for use in pediatric patients. Alternatively, if the answer cannot be determined without more information, please say that and explain what additional information is needed. Otherwise, no explanation is needed. Provide your response in a message to the console. Please consider the following additional sources when responding: 1. 21 CFR 201.128.pdf 2. Guidance from FDA on Unapproved Uses.pdf

    Expected output: message_in_console
  13. World420_RO_01 (task_bdd8216b4e864b9a81d0b9d7af624497) primary
    Law · Law World 420 (world_85a3713cd2794fdfb56e92161325a00e)

    Livyra wants to market Bencontra using just its name and price. What is the correct type of advertisement Livyra can make, if any, and what conditions must the advertisement meet? Answer in one sentence as a message in console. Use the folder Bencontra Data, the folder Legal Authorities, and "21 CFR Part 202.pdf" in your analysis.

    Expected output: message_in_console
  14. World 420 LB_05 (task_fa15dfc96f4b4a289fac616a6f3b4a1a) primary
    Law · Law World 420 (world_85a3713cd2794fdfb56e92161325a00e)

    For purposes of a training presentation for sales representatives giving background on what off-label promotion is and its legal status, please indicate whether it would be accurate to incorporate the following items from Livyra_handbook.pdf: (1) The definition of "misbranding" (2) The language in section 2.3 Please note that you should not consider the above items inaccurate solely based on being incomplete since additional language will be added for purposes of the training session. Also, there is no need to explain your reasoning; I just need a "would be accurate" or "would not be accurate" answer for each item. Provide your response in a message to the console. Consider the following additional sources: 1. US v Facteau.pdf 2. 21 USC 331.pdf 3. 21 USC 352.pdf

    Expected output: message_in_console
  15. Task p5y453db (task_48d70f58c409446fb1e22ca7572dd1c8) primary
    Law · Law World 420 (world_85a3713cd2794fdfb56e92161325a00e)

    Livyra has scrapped the "Focus on What Matters" campaign. They intend to replace it with a new campaign advertising to the general population which states normal approved audio messages based on depression treatment in adults but is supplemented with images of happy children hugging their parents. Looking at the guidance from the FDA, "Guidance from FDA on Unapproved Uses.pdf", as well as case law such as US v. Caronia, "US v. Caronia, 703 F. 3d 149 - Court of Appeals, 2nd Circuit 2012.pdf", will Livyra be in violation of FDA regulations regarding off-label marketing?

    Expected output: message_in_console
  16. World 420_LB_06 (task_3ff7f71e606840d09b4d64bd88398799) primary
    Law · Law World 420 (world_85a3713cd2794fdfb56e92161325a00e)

    You are an attorney. Please review the document regarding circulation issues associated with Bencontra. Does Livyra's dissemination of this document to a doctor who prescribes Bencontra appear to meet the standard for misbranding under 21 U.S.C. 352(a)? And does it appear to provide support for a misbranding finding under 21 U.S.C. 352(f)? Please give a "yes" or "no" answer to these questions; you do not need to elaborate on your reasoning. Output your answers as a message in the console. Also, for purposes of these questions, you should assume that (1) the document is being distributed in response to a doctor's question about circulation issues associated with Bencontra and (2) any factual information in the document is true. In addition to the the document regarding circulation, please consider the following sources: 1. 21 U.S.C. 352 2. FDA approval letter for Bencontra 3. 21 CFR 202.1 4. FDA First Amendment memo

    Expected output: message_in_console
  17. Task 2cc51b31 (task_cdaf73e0d7ce466a8338793974c13d22) primary
    Law · Law World 420 (world_85a3713cd2794fdfb56e92161325a00e)

    A doctor was recently charged by the federal government under the same regulations that were the subject of the case in "US v. Caronia, 703 F. 3d 149 - Court of Appeals, 2nd Circuit 2012.pdf" for prescribing Bencontra to a 15-year-old patient to help with ADHD, which has shown promise in studies. Analyzing "US v. Caronia, 703 F. 3d 149 - Court of Appeals, 2nd Circuit 2012.pdf", what is the doctor's best legal defense?

    Expected output: message_in_console
  18. World 420_dds_03 (task_8b881ab96d114632bfdbe3ad35684f33) primary
    Law · Law World 420 (world_85a3713cd2794fdfb56e92161325a00e)

    Prepare an analysis regarding Livyra's Bencontra promotional activities. Take a look at the case file, paying close attention to the email communications as well as the relevant case law, statutes and other legal guidance. Also, use the attached document. Please provide a brief overview of the relevant laws then address whether the company faces potential liability under the law by identifying emails and violations at issue. Output your assessment as a message to me here.

    Expected output: message_in_console
  19. World 420_ZT_02 (task_4fe676ac270b404c9104c758bd3964dc) primary
    Law · Law World 420 (world_85a3713cd2794fdfb56e92161325a00e)

    Review the attached email (stored as a docx file), as well as the cases available in the world. Write a message to me, outlining whether John’s actions constitute unlawful off-label promotion.

    Expected output: message_in_console
  20. World420_SF_02 (task_d06f10ebd2ad4bb19ba47b00a590dd54) primary
    Law · Law World 420 (world_85a3713cd2794fdfb56e92161325a00e)

    Livyra has decided to pay its independent contractors a fixed salary, consistent with fair market value, to sell Bencontra that is payable by government healthcare. Each independent contractor is given a written 1-year contract. Does this likely violate the anti-kickback statute? Please reply to me with a single sentence answer and explanation.

    Expected output: message_in_console

Public transcript

Task transcript