How to Use the AQASR

Following an introduction that relies on a flowchart (see Figure 1) to lay out the typical process of conducting a systematic review, this document offers a list of questions that systematic review users should ask themselves. For each question, there is a list of the type of information to look for in answering it and an explanation of why the question is important (termed rationale). The AQASR checklist is a separate document that lists the same questions (without the items to look for and the rationale) and gives the user space to note their observations on a particular systematic review.

A core set of questions can be asked of every systematic review, whether it deals with prevention studies or economic evaluation of treatment studies. Not all questions in these sections are relevant to all systematic reviews. This document covers questions relevant only to meta-analysis, a genre of systematic review that attempts to provide a quantitative synthesis of the literature. These questions are presented in the following sections:

  1. Systematic review question/clinical applicability (RQ)
  2. Protocol (PR)
  3. Database searching (DB)
  4. Other searches (OS)
  5. Search limitations (SL)
  6. Abstract and full paper scanning (SC)
  7. Methodological quality assessment and use (MQ)
  8. Data extracting (DA)
  9. Qualitative synthesis (QS)
  10. Discussion (DI)
  11. Various (VA)
  12. Meta-analysis (MA)

The following sections of the AQASR checklist provide questions for five types of systematic reviews that the Task Force identified as most salient to rehabilitation decision makers, including:

  1. intervention studies, including all treatments and preventive measures (IN)
  2. prognostic studies (PS)
  3. diagnostic accuracy studies (DS)
  4. studies of the quality of measurement instruments (MI)
  5. economic evaluations (EC)
Whether a particular question is relevant to the issue at hand (which is always, “Can I rely on the conclusions and recommendations this review provides?”) depends in part on one’s purpose in reading the review: “What actions potentially need to be taken, modified, or omitted on the basis of the results?” The relevancy also may depend in part on the nature of the review—for instance, the limitations the authors imposed on the scope and method of their review. For these reasons, a numerical scoring system is not practical and would not provide useful results. The user can give more weight to specific characteristics, depending on their unique situation for potentially applying information from a particular review. There are many possible ways to use the AQASR checklist. Initially, you may want to write either an answer or “N/A” (not applicable) in every answer box, forcing yourself to read and reread the systematic review until all questions are answered. As you become more familiar with the critical reading of systematic reviews, you may want to use the AQASR checklist to make notes on particularly problematic issues only. There may come a time when you have become so adept at reading systematic reviews and extracting all information that bears on the quality and “dependability” of a review that you only need to review the checklist once to confirm that you have not skipped any important question in your mental appraisal of the review article.