Medical writers occupy a position unlike any other in the pharmaceutical industry. You are often the last person to read the evidence critically before it reaches regulators, healthcare professionals, or patients. A clinical study report passes through statisticians who focus on numbers, clinicians who focus on outcomes, and project managers who focus on timelines — but it is the medical writer who must read the entire body of evidence and render it into a coherent, defensible narrative.

This is not a passive role. If the evidence is weak, biased, or incomplete, a medical writer who simply transcribes the data has failed in their professional responsibility. Critical appraisal is the skill that separates a medical writer from a medical typist.

The consequences of uncritical evidence handling are real and measurable:

• Regulatory rejection: Agencies like the EMA and FDA employ expert reviewers trained in clinical epidemiology. They will identify methodological flaws you missed — and the resulting delay can cost a sponsor millions and months of market exclusivity.
• Publication retraction: A systematic review or manuscript built on uncritically accepted source studies can collapse when peer reviewers or post-publication scrutiny exposes the underlying bias.
• Clinical harm: If a flawed study overstates efficacy or understates risk, the downstream documents you write — prescribing information, clinical guidelines, patient materials — can lead to real harm to patients.
• Career risk: Medical writers who consistently produce work that fails regulatory or peer review scrutiny do not advance. Critical appraisal is a career-differentiating skill.

Critical appraisal applies across every domain of medical writing:

• Publications: Appraising source studies for systematic reviews, checking the evidence base before writing a narrative review or manuscript
• Regulatory: Evaluating pivotal trials for clinical study reports, integrated summaries (ISS/ISE), and benefit-risk assessments
• Pharmacovigilance: Assessing signal-detection studies, case series, and real-world evidence for periodic safety reports
• HEOR: Evaluating clinical inputs for cost-effectiveness models, network meta-analyses, and HTA dossiers
• Medical affairs: Grading evidence for standard response letters, scientific platforms, and fair-balanced slide decks

In every case, the question is the same: Can I trust this evidence enough to build a professional document on it?

Every piece of evidence you encounter as a medical writer should be filtered through three fundamental questions:

1. Is the study valid? — Were the methods rigorous enough to produce trustworthy results? (Internal validity)
2. What are the results? — How large is the effect, how precise is the estimate, and is it clinically meaningful — not just statistically significant?
3. Can I apply this? — Does the study population, setting, and intervention match the context of the document I am writing? (External validity/generalisability)

These three questions — borrowed from the evidence-based medicine tradition — will recur throughout this course. Every tool you learn (CONSORT, RoB 2, GRADE) is a structured way to answer one or more of them.

The evidence hierarchy, most commonly associated with the Oxford Centre for Evidence-Based Medicine (CEBM), ranks study designs by their susceptibility to bias:

• Level 1: Systematic reviews and meta-analyses of randomised controlled trials
• Level 2: Individual randomised controlled trials (with narrow confidence intervals)
• Level 3: Cohort studies (prospective and retrospective)
• Level 4: Case-control studies, case series
• Level 5: Expert opinion, mechanism-based reasoning

The logic is straightforward: randomisation controls for confounding, so designs that include it are less susceptible to systematic error. Systematic reviews aggregate multiple studies, further reducing the influence of any single biased result.

The evidence pyramid works well for common diseases with large patient populations and straightforward interventions. It breaks down in several important situations that medical writers encounter regularly:

• Rare diseases: When a condition affects fewer than 5 in 10,000 people (the EU orphan threshold), randomising 500 patients may be impossible. A well-conducted single-arm trial with a historical control may be the best feasible evidence.
• Ethical constraints: You cannot randomise patients to a known harmful exposure. The evidence that smoking causes lung cancer comes from cohort and case-control studies — not RCTs.
• Surgical and device interventions: Blinding a surgical procedure is extremely difficult. Many device approvals rely on registry data and single-arm studies with performance goals.
• Urgent public health situations: During pandemics, observational data may be the only evidence available for months. Waiting for RCTs while people die is not always acceptable.
• Long-term outcomes: An RCT with 2-year follow-up cannot capture outcomes that take 20 years to manifest. Epidemiological cohorts fill this gap.

The CONSORT (Consolidated Standards of Reporting Trials) statement is typically taught as a reporting guideline — a checklist for authors writing up trial results. But for medical writers engaged in critical appraisal, CONSORT is equally powerful as a reading tool.

Every item on the CONSORT checklist corresponds to a piece of information you need to evaluate a trial's validity. If the information is missing from a publication, that itself is a red flag — either the authors forgot to report it (poor reporting) or the information would have revealed a methodological weakness (potential bias).

The key CONSORT items for critical reading are:

• Randomisation sequence generation: How was the random sequence created? (Computer-generated is ideal; coin-flipping or alternation is not truly random)
• Allocation concealment: Was the assignment hidden from investigators until the moment of enrolment? (Central randomisation via IVRS/IWRS is gold standard)
• Blinding: Who was blinded — participants, care providers, outcome assessors? Was blinding maintained?
• Participant flow: The CONSORT flow diagram showing enrolment, allocation, follow-up, and analysis numbers
• Analysis population: ITT, modified ITT, or per-protocol — and justification

Randomisation and allocation concealment are different concepts that are frequently confused:

Randomisation is the method used to generate the assignment sequence (e.g., computer-generated random numbers, block randomisation with varying block sizes). It ensures each participant has a known probability of being assigned to each group.

Allocation concealment is the mechanism that prevents investigators from knowing the next assignment before enrolling a patient. Without it, investigators can selectively enrol patients based on which group they would be assigned to — even if the sequence itself is random.

A trial can have adequate randomisation but inadequate allocation concealment. For example, if a randomisation list is generated by computer but kept in an unsealed envelope at the clinical site, an investigator can peek ahead and decide whether to enrol the next patient based on their likely assignment. This introduces selection bias despite proper randomisation.

Blinding (also called masking) prevents knowledge of treatment assignment from influencing outcomes. There are three levels:

• Single-blind: Participants do not know their assignment (but investigators do)
• Double-blind: Neither participants nor investigators know the assignment
• Triple-blind: Participants, investigators, and outcome assessors are all blinded

The importance of blinding depends on the type of outcome. For subjective outcomes (pain scores, quality of life, physician global assessment), blinding is critical because knowledge of assignment directly influences reporting. For objective outcomes (all-cause mortality, laboratory values), blinding matters less — though it can still influence decisions about treatment discontinuation, co-interventions, and diagnostic workup.

Some interventions make blinding difficult or impossible: surgery vs medical therapy, physiotherapy vs no treatment, device vs drug. In these cases, blinded outcome assessment (PROBE design: Prospective Randomised Open-label, Blinded Endpoint evaluation) is a practical alternative.

The choice of analysis population fundamentally affects the result — and the interpretation:

Intention-to-treat (ITT): Includes all randomised patients, analysed in the group to which they were assigned, regardless of protocol adherence. ITT preserves the benefit of randomisation and reflects real-world effectiveness. It is generally conservative (biased toward the null) for superiority trials.

Per-protocol (PP): Includes only patients who completed the study per protocol — excluding dropouts, protocol violations, and poor adherers. PP is biased because the reasons for exclusion are rarely random. Sicker patients drop out more, non-responders discontinue, and adverse events cause withdrawals. PP analyses tend to overestimate treatment effects.

Modified ITT (mITT): A common but controversial middle ground. Excludes patients who never received any treatment or had no post-baseline assessment. Accepted by regulators in some contexts but must be pre-specified and justified.

Observational studies — cohort, case-control, and cross-sectional designs — lack the one feature that makes RCTs powerful: randomisation. Without random assignment, the groups being compared may differ systematically in ways that influence the outcome. These systematic differences are called confounders.

The STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) checklist provides a structured framework for evaluating the reporting quality of observational studies. Like CONSORT for RCTs, STROBE is primarily a reporting guideline — but it doubles as a critical reading checklist.

Key STROBE items for critical reading include:

• Study design: Is it cohort, case-control, or cross-sectional? Each has different bias profiles.
• Setting and participants: How were participants selected? What were the eligibility criteria?
• Variables: How were exposure, outcome, and confounders defined and measured?
• Bias: What efforts were made to address potential sources of bias?
• Study size: How was the sample size determined?
• Statistical methods: How were confounders controlled? What sensitivity analyses were performed?

Medical writers need to recognise and name the specific biases that threaten observational study validity:

Confounding occurs when a third variable is associated with both the exposure and the outcome, creating a spurious association. For example, patients who exercise regularly may be prescribed Drug Z more often (confounding by indication) and also have lower mortality due to fitness — the apparent drug benefit is actually a fitness benefit.

Immortal time bias arises when follow-up is misclassified. If the treated group requires survival to a certain point (e.g., filling a second prescription), the time before that point is "immortal" — the patient cannot have the outcome during that period by definition. This systematically advantages the treated group.

Channeling bias (also called allocation bias) occurs when prescribing patterns differ by patient characteristics. Sicker patients may receive one drug while healthier patients receive another, creating a systematic difference between groups that mimics (or masks) a treatment effect.

Selection bias occurs when the study population is not representative of the target population, or when inclusion/exclusion criteria systematically differ between groups.

Information bias (measurement bias) occurs when exposure or outcome data is collected differently or inaccurately between groups. In retrospective studies using medical records, differential documentation quality can create this bias.

Confounding by indication is the most important bias in pharmacoepidemiological studies and one that medical writers encounter constantly. It occurs because drugs are prescribed for reasons — and those reasons are themselves associated with outcomes.

Consider a study comparing outcomes between patients who received an anticoagulant and those who did not. Patients who received the anticoagulant had atrial fibrillation (the indication). Patients who did not receive it may have been healthier, had lower stroke risk, or had contraindications to anticoagulation. Comparing these groups as if they were exchangeable produces a biased estimate.

Methods to address confounding by indication include multivariable regression (adjusting for measured confounders), propensity score methods (matching, weighting, or stratification), and instrumental variable analysis. However, none of these methods can control for unmeasured confounders — this is the fundamental limitation of observational research.

The Cochrane Risk of Bias 2 (RoB 2) tool is the current standard for assessing the risk of bias in randomised controlled trials. It replaced the original Cochrane Risk of Bias tool in 2019. RoB 2 is used in Cochrane reviews, NICE technology appraisals, WHO guidelines, and many peer-reviewed journals.

RoB 2 evaluates bias through five domains, each addressing a different mechanism through which bias can enter a trial result:

1. Domain 1: Bias arising from the randomisation process
2. Domain 2: Bias due to deviations from intended interventions
3. Domain 3: Bias due to missing outcome data
4. Domain 4: Bias in measurement of the outcome
5. Domain 5: Bias in selection of the reported result

Each domain contains signalling questions — specific yes/no/probably/no information questions that guide the assessor through a structured judgment. The answers to the signalling questions map to a domain-level judgment: Low risk, Some concerns, or High risk of bias.

RoB 2 uses a structured algorithm to move from signalling questions to domain judgments and from domain judgments to an overall judgment:

• Overall Low risk of bias: All five domains are judged as Low risk
• Overall Some concerns: At least one domain has Some concerns, but no domain is High risk
• Overall High risk of bias: At least one domain is High risk, OR multiple domains have Some concerns in a way that substantially lowers confidence in the result

Important: the overall judgment is not a vote count. A single High-risk domain is sufficient for an overall High-risk judgment. And multiple "Some concerns" judgments can collectively warrant High risk if the concerns are cumulative.

ROBINS-I (Risk Of Bias In Non-randomised Studies — of Interventions) is the Cochrane-endorsed tool for assessing bias in observational studies that estimate the effect of an intervention. It was designed as a companion to RoB 2, but with a critical philosophical difference.

The key difference: RoB 2 starts from the assumption that a well-conducted RCT has low risk of bias (bias must be demonstrated). ROBINS-I starts from the assumption that a non-randomised study has inherently high risk of bias due to confounding, and the study must demonstrate that it adequately addressed confounding to be downgraded to moderate or low risk.

ROBINS-I evaluates bias across seven domains:

1. Confounding — Were important confounders identified and controlled?
2. Selection of participants — Was selection into the study related to both intervention and outcome?
3. Classification of interventions — Were interventions clearly defined and consistently classified?
4. Deviations from intended interventions — Were there co-interventions, crossover, or non-adherence?
5. Missing data — Was outcome data available for an adequate proportion of participants?
6. Measurement of outcomes — Could outcome assessment have been influenced by knowledge of intervention?
7. Selection of the reported result — Was the reported result selected from multiple analyses?

QUADAS-2 (Quality Assessment of Diagnostic Accuracy Studies — version 2) is the tool for assessing bias in studies that evaluate the accuracy of a diagnostic test (sensitivity, specificity, positive/negative predictive value).

Diagnostic accuracy studies have unique bias concerns that RoB 2 and ROBINS-I do not address:

• Verification bias: Not all patients receive the reference standard (gold standard test). If only patients who tested positive on the index test receive the reference standard, sensitivity is overestimated.
• Spectrum bias: The study includes only patients with advanced disease or clearly healthy controls — overstating the test's ability to discriminate in the "grey zone" of early or equivocal disease.
• Review bias: The person interpreting the index test knows the reference standard result (or vice versa), contaminating the assessment.

QUADAS-2 assesses four domains: patient selection, index test, reference standard, and flow and timing. Each domain has signalling questions and risk-of-bias judgments, plus an applicability concern assessment for the first three domains.

The choice of bias assessment tool depends on the study design and the review question:

• RCTs: Use RoB 2
• Non-randomised studies of interventions (cohort, case-control, before-after studies evaluating a treatment): Use ROBINS-I
• Diagnostic accuracy studies: Use QUADAS-2
• Prognostic studies: Use QUIPS (Quality In Prognosis Studies)
• Prevalence and incidence studies: Use JBI or Hoy tools

Using the wrong tool produces misleading assessments. Applying RoB 2 to an observational study would fail to capture confounding — the most important bias in non-randomised research. Applying ROBINS-I to a diagnostic accuracy study would miss verification bias and spectrum bias entirely.

Unlike therapeutic trials that measure clinical outcomes over time (e.g., survival, disease progression, symptom relief), diagnostic accuracy studies evaluate how well an index test correctly classifies individuals as having or not having a target condition compared with a reference standard (“gold standard”).

Diagnostic studies are vulnerable to unique methodological distortions that tools like RoB 2 or ROBINS-I cannot evaluate:

• Spectrum Bias (Case-Control Distortion): Testing an assay only in severe, unmistakable cases and completely disease-free healthy controls artificially inflates both sensitivity and specificity. In real-world clinical practice, tests must distinguish disease from subtle, clinically ambiguous mimickers.
• Partial / Differential Verification Bias: Occurs when only patients who test positive on the index test receive the definitive reference standard (e.g., invasive tissue biopsy), while negative patients receive only clinical follow-up.
• Incorporation Bias: Occurs when the index test under evaluation is actually part of the reference standard criteria used to define disease status.

The Grading of Recommendations, Assessment, Development and Evaluations (GRADE) framework is the most widely adopted system for rating the certainty (also called quality) of a body of evidence and the strength of clinical recommendations. It is used by the WHO, Cochrane, NICE, the EMA (for benefit-risk assessment frameworks), and over 100 other organisations worldwide.

GRADE answers a specific question: How confident are we that the true effect lies close to the estimated effect? This is not a question about a single study — it is a question about the entire body of evidence for a specific outcome.

GRADE produces four levels of certainty:

• High: We are very confident that the true effect lies close to the estimate. Further research is very unlikely to change our confidence.
• Moderate: We are moderately confident. The true effect is likely to be close to the estimate, but there is a possibility it is substantially different.
• Low: Our confidence is limited. The true effect may be substantially different from the estimate.
• Very Low: We have very little confidence. The true effect is likely to be substantially different from the estimate.

GRADE begins with a starting rating determined by study design:

• RCTs start at High certainty — because randomisation controls confounding
• Observational studies start at Low certainty — because confounding is inherent

From this starting point, the certainty can be rated down for five reasons or rated up for three reasons.

1. Risk of bias — If the studies in the body of evidence have methodological flaws (as assessed by RoB 2 or ROBINS-I), the certainty is rated down. Rate down by one level for serious risk of bias, two levels for very serious risk.

2. Inconsistency — If the results across studies point in different directions (high heterogeneity, wide variation in effect sizes), the certainty is rated down. Assessed using I², visual inspection of the forest plot, and consideration of whether subgroup analyses explain the heterogeneity.

3. Indirectness — If the evidence does not directly address the clinical question — different population, different intervention, different comparator, different outcome, or different setting — the certainty is rated down. This is the applicability question.

4. Imprecision — If the confidence interval around the effect estimate is wide enough to include clinically important differences in both directions (benefit and harm), or if the sample size is too small for the optimal information size (OIS), the certainty is rated down.

5. Publication bias — If there is evidence that studies with negative or null results were not published (assessed through funnel plot asymmetry, trial registration checks, or small-study effects), the certainty is rated down.

These apply primarily to observational evidence (which starts at Low):

1. Large magnitude of effect — A very large effect (RR >2 or <0.5 with no plausible confounders) can be rated up by one level; a very very large effect (RR >5 or <0.2) can be rated up by two levels. The logic: an effect this large is unlikely to be entirely explained by confounding.

2. Dose-response gradient — If there is a clear, monotonic relationship between dose/exposure and outcome, this strengthens causal inference and can support rating up.

3. Plausible confounding would reduce the effect — If all plausible confounders would bias the result toward the null (i.e., the true effect is likely larger than observed), this can support rating up.

Publication bias is the systematic tendency for studies with positive, significant, or "interesting" results to be published more frequently, more quickly, and in higher-impact journals than studies with negative, null, or inconclusive results. It is one of the most insidious threats to evidence synthesis because it affects what you can find, not just what you find.

The consequences are severe: if a meta-analysis pools only the published (positive) studies, the pooled effect estimate will be systematically inflated. The true effect may be smaller, or even null, but the unpublished negative studies are invisible.

Publication bias differs from reporting bias, which refers to selective reporting within a study (e.g., reporting only favourable outcomes, changing the primary endpoint post hoc). Both distort the evidence base, but publication bias operates at the level of entire studies, while reporting bias operates at the level of outcomes within a study.

Several methods exist to detect publication bias, though none is definitive:

Funnel plots: Plot the effect size (x-axis) against the study precision (y-axis, usually standard error). In the absence of publication bias, the plot should be roughly symmetrical — small studies (imprecise) scatter widely around the pooled estimate, while large studies (precise) cluster tightly. Asymmetry — a gap where small, non-significant studies should be — suggests publication bias.

Egger's test: A statistical test for funnel plot asymmetry. A significant Egger's test (p<0.1 is the conventional threshold) provides evidence of small-study effects, which may indicate publication bias. However, Egger's test has low power when there are fewer than 10 studies.

Trim-and-fill method: A statistical technique that estimates the number of "missing" studies, imputes them, and recalculates the pooled effect. It provides an adjusted estimate that accounts for the suspected missing studies. However, it assumes publication bias is the only cause of asymmetry.

Trial registration comparison: Comparing registered trials (ClinicalTrials.gov, PROSPERO, EU Clinical Trials Register) with published trials. If many registered trials have no corresponding publication, this is direct evidence of non-publication.

The primary countermeasure against publication bias is prospective trial registration. Since 2005, the International Committee of Medical Journal Editors (ICMJE) has required prospective registration as a condition of publication. ClinicalTrials.gov, the EU Clinical Trials Register, and the WHO ICTRP serve as public registries.

For systematic review protocols, PROSPERO serves a similar function — registering the review question, search strategy, and planned analyses before the review is conducted.

For medical writers, the practical implications are:

• Always search trial registries as part of your systematic review, not just bibliographic databases
• Compare registered outcomes with published outcomes to detect outcome reporting bias
• Include a funnel plot and Egger's test in any meta-analysis with 10+ studies
• Discuss the potential impact of publication bias in the limitations section of every systematic review

A forest plot is the standard visual display of a meta-analysis. Understanding its components is essential for critical reading:

• Point estimates: Each horizontal line represents one study. The square marks the point estimate (effect size). The size of the square indicates the weight of the study in the meta-analysis (larger square = more weight).
• Confidence intervals: The horizontal line through each square shows the 95% CI. Longer lines = less precise studies.
• Line of no effect: The vertical line at 1.0 (for ratio measures) or 0 (for difference measures). Studies whose CI crosses this line are individually non-significant.
• Diamond: The pooled estimate. The centre is the point estimate; the width is the 95% CI. If the diamond does not touch the line of no effect, the pooled result is statistically significant.
• Weight column: Shows each study's percentage contribution to the pooled estimate. Dominated by larger, more precise studies in fixed-effect models; more evenly distributed in random-effects models.

Heterogeneity refers to variability in the true treatment effects across studies — not just random sampling variability. When studies genuinely differ in their treatment effects, pooling them into a single number can be misleading.

Three measures of heterogeneity are commonly reported:

I² (I-squared): The percentage of variability across studies that is due to true differences rather than chance. Conventionally: I²<25% = low, 25–75% = moderate, >75% = high. But I² is influenced by the precision of the studies — large, precise studies can produce high I² even with small absolute differences.

Tau² (tau-squared): The estimated variance of the true effect sizes across studies. Unlike I², tau² is on the scale of the effect measure and gives an absolute measure of heterogeneity. More useful for understanding the actual spread of effects.

Cochran's Q: A chi-squared test for heterogeneity. A significant Q (p<0.10 is the conventional threshold, more liberal than usual because the test has low power) indicates statistically significant heterogeneity. But non-significance does not prove homogeneity — it may simply reflect low power.

Subgroup analyses in meta-analyses are common but frequently misleading. The ICEMAN (Instrument for assessing the Credibility of Effect Modification Analyses in randomised controlled trials and meta-analyses) criteria help evaluate whether a subgroup effect is real or a chance finding:

• Was the subgroup variable pre-specified? — Post-hoc subgroups are hypothesis-generating, not confirmatory.
• Was the direction of the subgroup effect pre-specified? — Predicting which direction the effect would differ is stronger than simply looking for any difference.
• Is there a biological or clinical rationale? — A plausible mechanism strengthens credibility.
• Is the subgroup analysis one of a small number of analyses? — Testing 20 subgroups guarantees spurious findings by chance alone.
• Is the interaction test significant? — Comparing subgroup point estimates is insufficient; a formal test of interaction is required.

Traditional pairwise meta-analysis combines studies directly comparing Treatment A versus Treatment B. However, in modern pharmaceutical development, multiple competing drugs exist, but direct head-to-head RCTs between them are rarely conducted.

Network Meta-Analysis (NMA) (also called multiple treatments meta-analysis) synthesizes both direct evidence (trials directly comparing A vs B) and indirect evidence (trials comparing A vs Placebo and B vs Placebo) across an entire evidence network.

While PRISMA 2020 is a reporting guideline that tells authors what to report in a systematic review manuscript, AMSTAR 2 is a critical appraisal instrument that evaluates how rigorously the review was conducted.

AMSTAR 2 evaluates 16 distinct domains, with 7 critical domains that directly determine whether the overall confidence in the review results is High, Moderate, Low, or Critically Low.

Real-world evidence (RWE) derives from real-world data (RWD) — data collected outside of conventional clinical trials, including electronic health records, claims databases, disease registries, and patient-generated data. Both the EMA and FDA have published frameworks for using RWE in regulatory decision-making.

RWE has genuine strengths: it reflects routine clinical practice, captures long-term outcomes, includes diverse populations, and can study questions that RCTs cannot (rare outcomes, comparative effectiveness of marketed products, long-term safety).

But RWE also inherits all the biases of observational research — confounding, selection bias, information bias, immortal time bias — compounded by data quality issues inherent to administrative and clinical databases. The critical appraisal challenge is distinguishing RWE that provides genuine causal evidence from RWE that merely shows association dressed up as causation.

The target trial emulation framework, developed by Hernán and Robins, is the gold standard for designing and appraising RWE studies. The idea is simple: before analysing observational data, define the hypothetical randomised trial (the target trial) that you would conduct if it were feasible. Then design your observational analysis to emulate each component of that target trial as closely as possible.

The components to emulate include:

• Eligibility criteria: Who would be enrolled in the target trial?
• Treatment strategies: What are the interventions being compared?
• Assignment: How would patients be randomised? (In the observational study: how is treatment assignment mimicked?)
• Start of follow-up: When does time zero begin? (Crucial for avoiding immortal time bias)
• Outcome: What is measured and when?
• Causal contrast: ITT or per-protocol effect?
• Statistical analysis: How are confounders handled?

When appraising RWE, ask: Did the study explicitly define and emulate a target trial? Studies that do are generally more rigorous than those that simply "compare outcomes between groups."

A directed acyclic graph (DAG) is a visual tool for mapping the causal relationships between variables in a study. It helps identify which variables are confounders (must be adjusted for), mediators (should not be adjusted for), and colliders (adjusting for them introduces bias).

A simple DAG for a drug-outcome study might look like:

Disease severity → Treatment choice (sicker patients get the drug)
Disease severity → Hospitalisation (sicker patients are hospitalised more)
Treatment → Hospitalisation (the causal effect we want to estimate)

In this DAG, disease severity is a confounder because it affects both treatment choice and the outcome. If it is not measured and adjusted for, the estimated treatment effect is biased.

DAGs are particularly useful for identifying unmeasured confounders — variables that are causally relevant but not available in the database. A well-constructed DAG can reveal whether the available adjustment variables are sufficient to produce an unbiased estimate, or whether residual confounding is inevitable.

In evidence-based medicine, demonstrating that a drug is clinically effective and safe is only the first step. Healthcare payers and reimbursement authorities (e.g., NICE in the UK, CADTH in Canada, PBAC in Australia, G-BA in Germany) require proof that the therapy represents an efficient use of finite healthcare resources.

Medical writers working on publications, HTA dossiers, and value dossiers must evaluate the three major types of economic analyses:

• Cost-Effectiveness Analysis (CEA): Measures outcomes in natural clinical units (e.g., cost per life-year gained, cost per exacerbation avoided).
• Cost-Utility Analysis (CUA): Measures outcomes in Quality-Adjusted Life Years (QALYs), combining length of life and health-related quality of life (HRQoL utility weights from 0.0 [death] to 1.0 [perfect health]). Summarized as the Incremental Cost-Effectiveness Ratio (ICER): ICER = (CostNew − CostStandard) / (QALYNew − QALYStandard)
• Budget Impact Analysis (BIA): Predicts the total financial impact of introducing the new drug on a specific healthcare budget over a 1-to-5 year horizon.

In the publications domain, critical appraisal is most visible in systematic reviews and meta-analyses. The medical writer must:

• Apply RoB 2 or ROBINS-I to each included study
• Use GRADE to rate the certainty of evidence for each outcome
• Construct Summary of Findings tables
• Interpret forest plots, heterogeneity, and prediction intervals
• Write balanced discussion sections that accurately reflect the strength and limitations of the evidence

For individual manuscripts, critical appraisal ensures that the discussion section accurately contextualises the study's findings within its limitations — not overstating results or ignoring methodological weaknesses.

In the regulatory domain, critical appraisal underpins benefit-risk assessment. Regulatory agencies (EMA, FDA, PMDA) evaluate whether the benefits of a product outweigh its risks — and this evaluation depends entirely on the quality of the underlying evidence.

Medical writers use appraisal skills when writing:

• Clinical study reports (CSRs): Ensuring the methods, results, and conclusions are internally consistent and transparently presented
• Integrated summaries (ISS/ISE): Synthesising data across multiple trials, requiring assessment of consistency and cross-trial comparability
• Benefit-risk frameworks: GRADE-like approaches are increasingly used (e.g., the EMA's PrOACT-URL framework) to structure the assessment of benefit and risk evidence
• Clinical evaluation reports (for medical devices): Appraising the literature and clinical investigation data to demonstrate device safety and performance under the EU MDR

Pharmacovigilance (PV): Signal evaluation relies on critical appraisal of case series, spontaneous reports, and epidemiological studies. Writers assess whether an observed signal represents a true causal association or is confounded, biased, or coincidental. Bradford Hill criteria (strength, consistency, temporality, etc.) are the appraisal framework in this domain.

HEOR (Health Economics and Outcomes Research): Evidence synthesis for HTA dossiers requires GRADE assessments, network meta-analyses, and critical appraisal of the clinical inputs that feed into economic models. If the clinical evidence is biased, the economic model inherits that bias — "garbage in, garbage out."

Medical affairs: Scientific exchange materials, medical information standard response letters, and advisory board backgrounds all require evidence grading. Medical affairs writers must assess the level and quality of evidence behind every claim to ensure fair balance and avoid unsupported assertions.

Medical devices: Clinical evaluation reports under the EU MDR require systematic appraisal of both clinical investigation data and equivalent device literature. The MEDDEV 2.7/1 Rev 4 guidance provides the appraisal framework for device evidence.

Medical information: Standard response documents grade evidence using frameworks specific to the medical information function (e.g., SORT — Strength of Recommendation Taxonomy, or custom internal grading scales). Every response must accurately reflect the quality of the evidence cited.

Read the following study abstract and methods excerpt carefully. Then complete each appraisal task below.