AMSTAR 2

Appraising the methodological quality of systematic reviews of healthcare interventions, including reviews that include randomized trials, non-randomized studies of interventions, or both. AMSTAR 2 is most useful when users need to decide how much confidence to place in the results of a published systematic review.

• Systematic reviews of randomized controlled trials of healthcare interventions
• Systematic reviews of non-randomized studies of interventions (NRSI)
• Systematic reviews that combine randomized and non-randomized study designs
• Mixed-methods systematic reviews where the intervention question can be evaluated against the AMSTAR 2 items

AMSTAR 2 is organised as a 16-item checklist. Seven of the items are designated as "critical domains" because weaknesses in those items can substantially undermine confidence in a review's conclusions. The remaining nine items are "non-critical": they affect quality but not as severely. The seven critical domains are listed below by their topic labels; for the actual signalling questions and the detailed response options, consult the official AMSTAR 2 paper and guidance document linked under Reference Documents.

1. 1.Protocol registered before commencement of the review (Item 2)

   Whether the review authors registered or otherwise made publicly available a protocol describing their methods before they began the review. A pre-specified protocol guards against post-hoc decisions that can introduce selection and reporting bias into the review process.

2. 2.Adequacy of the literature search (Item 4)

   Whether the review authors used a comprehensive literature search strategy that included multiple databases, supplementary search methods, and appropriate keywords and controlled vocabulary. An incomplete search risks missing relevant studies and introducing bias into the synthesised result.

3. 3.Justification for excluding individual studies (Item 7)

   Whether the review authors provided a list of studies they considered but excluded, with explicit reasons for each exclusion. Without a justified exclusion list, readers cannot verify whether the review captured the relevant evidence base.

4. 4.Risk of bias assessment for individual included studies (Item 9)

   Whether the review authors used a satisfactory technique for assessing the risk of bias in the individual studies included in the review. This is particularly important for reviews of non-randomized studies where confounding and selection bias are major concerns.

5. 5.Appropriateness of meta-analytical methods (Item 11)

   When meta-analysis was performed, whether the authors used appropriate statistical methods for combining results, including suitable handling of heterogeneity and choice of fixed-effect or random-effects models.

6. 6.Consideration of risk of bias when interpreting review results (Item 13)

   Whether the review authors took the risk of bias of individual included studies into account when interpreting and discussing the results of the review, rather than presenting the synthesised result as if all included studies were equally trustworthy.

7. 7.Assessment of presence and likely impact of publication bias (Item 15)

   Whether the review authors carried out an adequate investigation of publication bias (small-study effects), which can systematically distort the synthesised result if studies with non-significant or unfavourable results were less likely to be published.

Use AMSTAR 2 when you need to appraise the methodological quality of a published systematic review of a healthcare intervention. It is particularly appropriate when the review you are appraising includes non-randomized studies or a mixture of designs. AMSTAR 2 is widely used in umbrella reviews (reviews of systematic reviews), in evidence-based clinical guidelines that draw on existing systematic reviews, in health technology assessments, and in methods research that compares the quality of reviews across topic areas. Each AMSTAR 2 appraisal should be performed by at least two independent reviewers with disagreements resolved through discussion or arbitration.

When not to use it

AMSTAR 2 is not designed to appraise primary studies; for individual randomized trials use RoB 2 and for individual non-randomized cohort studies of interventions use ROBINS-I V2. AMSTAR 2 is also not the right tool for evaluating systematic reviews of diagnostic test accuracy studies, prognostic studies, qualitative research, or epidemiological exposures, all of which have their own dedicated appraisal instruments. Finally, the authors of AMSTAR 2 explicitly caution against using the tool to generate a numerical "score" for a review; the intended output is an overall confidence rating derived from the pattern of weaknesses across the critical and non-critical items.

AMSTAR 2 is one of several tools for appraising systematic reviews. ROBIS (Whiting et al., 2016) is an alternative risk-of-bias tool for systematic reviews developed contemporaneously; ROBIS focuses more directly on bias domains while AMSTAR 2 takes a broader methodological-quality perspective. For appraising the underlying primary studies referenced by a review, use RoB 2 (for randomized trials) or ROBINS-I V2 (for non-randomized cohort studies of interventions) instead. AMSTAR 2 and ROBIS are sometimes applied side by side in methods research to triangulate quality judgements. For comparing systematic reviews across multiple appraisal frameworks, CoRATES supports running parallel appraisals against the same body of evidence.

• AMSTAR 2 vs ROBIS: which tool for appraising systematic reviews?

CoRATES presents AMSTAR 2 as a structured digital workflow rather than a paper checklist or spreadsheet. Each of the 16 items is rendered with its response options in context, and the seven critical items are clearly distinguished from the nine non-critical items so reviewers know which weaknesses carry the most weight. Reviewers can attach rationale and direct quotes from the source review to each judgement. As items are completed, CoRATES applies the published decision rules to derive the overall confidence rating and updates it in real time. Multiple reviewers can complete independent AMSTAR 2 appraisals of the same review in parallel, and CoRATES surfaces disagreements in a side-by-side reconciliation view so they can be resolved before locking the appraisal. Completed appraisals can be exported as publication-ready summaries.

• Shea et al. (2017): AMSTAR 2 publication, BMJ
• AMSTAR 2 official website (amstar.ca)
• AMSTAR 2 guidance document (PDF)

The original AMSTAR was published in 2007 (Shea et al., BMC Medical Research Methodology) as an 11-item tool for appraising systematic reviews of randomized trials. AMSTAR 2 (Shea et al., BMJ 2017) is a substantive revision: the number of items expanded from 11 to 16, the scope expanded to include non-randomized studies of interventions, the tool introduced the explicit critical/non-critical domain distinction, and the overall confidence rating system (High / Moderate / Low / Critically Low) was added. Some items in the original AMSTAR were merged or split, and several new items were added to better address the methodological challenges of including non-randomized evidence.

• Treating AMSTAR 2 as a scoring system. The original authors are explicit that AMSTAR 2 is not designed to produce a numerical score; presenting a result as "13 out of 16" or as a percentage is a misuse of the tool. Report the overall confidence rating instead.
• Failing to distinguish critical from non-critical items. The seven critical items are designed to carry disproportionate weight in the overall judgement; treating all 16 items as equal will misrepresent the review's actual quality.
• Applying AMSTAR 2 to primary studies instead of systematic reviews. AMSTAR 2 appraises reviews; for individual randomized trials use RoB 2 and for non-randomized cohort studies use ROBINS-I V2.
• Using AMSTAR 2 for non-intervention reviews. The tool is scoped to reviews of healthcare interventions. Reviews of diagnostic test accuracy, prognostic factors, or epidemiological exposures should be appraised with tools designed for those study types.
• Single-reviewer appraisal. AMSTAR 2 appraisals should be carried out by at least two independent reviewers with disagreements resolved through discussion. Single-reviewer appraisals risk introducing the appraiser's own biases into the result.
• Neglecting the publication bias item. Item 15 is one of the seven critical items but is often overlooked in practice because publication bias assessment is technically demanding. A review that does not adequately investigate publication bias should not receive a "high" confidence rating.
• Confusing AMSTAR 2 with ROBIS. Both are appraisal tools for systematic reviews but they take different perspectives. AMSTAR 2 emphasises overall methodological quality while ROBIS focuses more directly on bias domains.

What does AMSTAR stand for?

AMSTAR is an acronym for "A MeaSurement Tool to Assess systematic Reviews". The "2" in AMSTAR 2 indicates the second major version, published in 2017 as a substantive revision and expansion of the original 2007 tool.

How is AMSTAR 2 different from the original AMSTAR?

AMSTAR 2 expanded the number of items from 11 to 16, broadened the scope from reviews of randomized trials only to reviews that include non-randomized studies of interventions, introduced the explicit distinction between critical and non-critical domains, and added the overall confidence rating system (High / Moderate / Low / Critically Low). Some original items were merged or split, and several new items were added to address the methodological challenges of including non-randomized evidence.

How many items does AMSTAR 2 have?

AMSTAR 2 has 16 items in total. Seven of these are designated as critical domains because weaknesses in those items can substantially undermine confidence in a review's conclusions. The remaining nine items are non-critical: they affect quality but not as severely as the critical items.

Does AMSTAR 2 produce a score?

No. The authors of AMSTAR 2 are explicit that the tool is not intended to generate an overall numerical score. Instead, AMSTAR 2 produces an overall confidence rating in the results of the review (High, Moderate, Low, or Critically Low), derived from the pattern of weaknesses across the critical and non-critical items.

Should I use AMSTAR 2 or ROBIS?

Both tools are valid options for appraising systematic reviews and they take complementary perspectives. AMSTAR 2 focuses on overall methodological quality while ROBIS focuses more directly on risk of bias in the review process. Some methods researchers apply both tools side by side to triangulate their judgements. The choice may also depend on which tool is required by the journal, funder, or guideline body you are reporting to.

Can AMSTAR 2 be used to appraise primary studies?

No. AMSTAR 2 is designed to appraise systematic reviews, not the individual studies included within them. To appraise individual randomized trials, use RoB 2. To appraise individual non-randomized cohort studies of interventions, use ROBINS-I V2.

How does CoRATES support AMSTAR 2 appraisals?

CoRATES presents the 16 AMSTAR 2 items in a structured digital workflow with critical items clearly distinguished from non-critical items. Reviewers can attach rationale and quotes from the source review to each judgement, and CoRATES applies the published decision rules to derive the overall confidence rating in real time. Multiple reviewers can complete independent appraisals of the same review in parallel; disagreements are surfaced for reconciliation before the appraisal is locked.

Where are the official AMSTAR 2 items and guidance?

The official AMSTAR 2 publication is freely available in the BMJ (Shea et al., BMJ 2017;358:j4008), and the AMSTAR website at amstar.ca hosts the checklist and a guidance document with item-level explanations. CoRATES does not reproduce the official content; reviewers should consult the official sources alongside their appraisals. See Reference Documents above for direct links.

Beverley J. Shea, Barnaby C. Reeves, George Wells, Micere Thuku, Candyce Hamel, Julian Moran, David Moher, Peter Tugwell, Vivian Welch, Elizabeth Kristjansson, and David A. Henry. The original publication appeared in the BMJ in September 2017 (Shea et al., BMJ 2017;358:j4008).

• Shea BJ, Reeves BC, Wells G, et al. (2017). AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ 2017;358:j4008. View
• Shea BJ, Grimshaw JM, Wells GA, et al. (2007). Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Medical Research Methodology, 7, 10. View
• Whiting P, Savovic J, Higgins JPT, et al. (2016). ROBIS: A new tool to assess risk of bias in systematic reviews was developed. Journal of Clinical Epidemiology, 69, 225-234. View