the 5 c's of literature review

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• How to Write a Literature Review | Guide, Examples, & Templates

How to Write a Literature Review | Guide, Examples, & Templates

Published on January 2, 2023 by Shona McCombes . Revised on September 11, 2023.

What is a literature review? A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research that you can later apply to your paper, thesis, or dissertation topic .

There are five key steps to writing a literature review:

• Search for relevant literature
• Evaluate sources
• Identify themes, debates, and gaps
• Outline the structure
• Write your literature review

A good literature review doesn’t just summarize sources—it analyzes, synthesizes , and critically evaluates to give a clear picture of the state of knowledge on the subject.

Table of contents

What is the purpose of a literature review, examples of literature reviews, step 1 – search for relevant literature, step 2 – evaluate and select sources, step 3 – identify themes, debates, and gaps, step 4 – outline your literature review’s structure, step 5 – write your literature review, free lecture slides, other interesting articles, frequently asked questions, introduction.

• Quick Run-through
• Step 1 & 2

When you write a thesis , dissertation , or research paper , you will likely have to conduct a literature review to situate your research within existing knowledge. The literature review gives you a chance to:

• Demonstrate your familiarity with the topic and its scholarly context
• Develop a theoretical framework and methodology for your research
• Position your work in relation to other researchers and theorists
• Show how your research addresses a gap or contributes to a debate
• Evaluate the current state of research and demonstrate your knowledge of the scholarly debates around your topic.

Writing literature reviews is a particularly important skill if you want to apply for graduate school or pursue a career in research. We’ve written a step-by-step guide that you can follow below.

A faster, more affordable way to improve your paper

Scribbr’s new AI Proofreader checks your document and corrects spelling, grammar, and punctuation mistakes with near-human accuracy and the efficiency of AI!

Proofread my paper

Writing literature reviews can be quite challenging! A good starting point could be to look at some examples, depending on what kind of literature review you’d like to write.

• Example literature review #1: “Why Do People Migrate? A Review of the Theoretical Literature” ( Theoretical literature review about the development of economic migration theory from the 1950s to today.)
• Example literature review #2: “Literature review as a research methodology: An overview and guidelines” ( Methodological literature review about interdisciplinary knowledge acquisition and production.)
• Example literature review #3: “The Use of Technology in English Language Learning: A Literature Review” ( Thematic literature review about the effects of technology on language acquisition.)
• Example literature review #4: “Learners’ Listening Comprehension Difficulties in English Language Learning: A Literature Review” ( Chronological literature review about how the concept of listening skills has changed over time.)

You can also check out our templates with literature review examples and sample outlines at the links below.

Download Word doc Download Google doc

Before you begin searching for literature, you need a clearly defined topic .

If you are writing the literature review section of a dissertation or research paper, you will search for literature related to your research problem and questions .

Make a list of keywords

Start by creating a list of keywords related to your research question. Include each of the key concepts or variables you’re interested in, and list any synonyms and related terms. You can add to this list as you discover new keywords in the process of your literature search.

• Social media, Facebook, Instagram, Twitter, Snapchat, TikTok
• Body image, self-perception, self-esteem, mental health
• Generation Z, teenagers, adolescents, youth

Search for relevant sources

Use your keywords to begin searching for sources. Some useful databases to search for journals and articles include:

• Your university’s library catalogue
• Google Scholar
• Project Muse (humanities and social sciences)
• Medline (life sciences and biomedicine)
• EconLit (economics)
• Inspec (physics, engineering and computer science)

You can also use boolean operators to help narrow down your search.

Make sure to read the abstract to find out whether an article is relevant to your question. When you find a useful book or article, you can check the bibliography to find other relevant sources.

You likely won’t be able to read absolutely everything that has been written on your topic, so it will be necessary to evaluate which sources are most relevant to your research question.

For each publication, ask yourself:

• What question or problem is the author addressing?
• What are the key concepts and how are they defined?
• What are the key theories, models, and methods?
• Does the research use established frameworks or take an innovative approach?
• What are the results and conclusions of the study?
• How does the publication relate to other literature in the field? Does it confirm, add to, or challenge established knowledge?
• What are the strengths and weaknesses of the research?

Make sure the sources you use are credible , and make sure you read any landmark studies and major theories in your field of research.

You can use our template to summarize and evaluate sources you’re thinking about using. Click on either button below to download.

Take notes and cite your sources

As you read, you should also begin the writing process. Take notes that you can later incorporate into the text of your literature review.

It is important to keep track of your sources with citations to avoid plagiarism . It can be helpful to make an annotated bibliography , where you compile full citation information and write a paragraph of summary and analysis for each source. This helps you remember what you read and saves time later in the process.

To begin organizing your literature review’s argument and structure, be sure you understand the connections and relationships between the sources you’ve read. Based on your reading and notes, you can look for:

• Trends and patterns (in theory, method or results): do certain approaches become more or less popular over time?
• Themes: what questions or concepts recur across the literature?
• Debates, conflicts and contradictions: where do sources disagree?
• Pivotal publications: are there any influential theories or studies that changed the direction of the field?
• Gaps: what is missing from the literature? Are there weaknesses that need to be addressed?

This step will help you work out the structure of your literature review and (if applicable) show how your own research will contribute to existing knowledge.

• Most research has focused on young women.
• There is an increasing interest in the visual aspects of social media.
• But there is still a lack of robust research on highly visual platforms like Instagram and Snapchat—this is a gap that you could address in your own research.

There are various approaches to organizing the body of a literature review. Depending on the length of your literature review, you can combine several of these strategies (for example, your overall structure might be thematic, but each theme is discussed chronologically).

Chronological

The simplest approach is to trace the development of the topic over time. However, if you choose this strategy, be careful to avoid simply listing and summarizing sources in order.

Try to analyze patterns, turning points and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred.

If you have found some recurring central themes, you can organize your literature review into subsections that address different aspects of the topic.

For example, if you are reviewing literature about inequalities in migrant health outcomes, key themes might include healthcare policy, language barriers, cultural attitudes, legal status, and economic access.

Methodological

If you draw your sources from different disciplines or fields that use a variety of research methods , you might want to compare the results and conclusions that emerge from different approaches. For example:

• Look at what results have emerged in qualitative versus quantitative research
• Discuss how the topic has been approached by empirical versus theoretical scholarship
• Divide the literature into sociological, historical, and cultural sources

Theoretical

A literature review is often the foundation for a theoretical framework . You can use it to discuss various theories, models, and definitions of key concepts.

You might argue for the relevance of a specific theoretical approach, or combine various theoretical concepts to create a framework for your research.

Like any other academic text , your literature review should have an introduction , a main body, and a conclusion . What you include in each depends on the objective of your literature review.

The introduction should clearly establish the focus and purpose of the literature review.

Depending on the length of your literature review, you might want to divide the body into subsections. You can use a subheading for each theme, time period, or methodological approach.

As you write, you can follow these tips:

• Summarize and synthesize: give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: don’t just paraphrase other researchers — add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically evaluate: mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: use transition words and topic sentences to draw connections, comparisons and contrasts

In the conclusion, you should summarize the key findings you have taken from the literature and emphasize their significance.

When you’ve finished writing and revising your literature review, don’t forget to proofread thoroughly before submitting. Not a language expert? Check out Scribbr’s professional proofreading services !

This article has been adapted into lecture slides that you can use to teach your students about writing a literature review.

Scribbr slides are free to use, customize, and distribute for educational purposes.

Open Google Slides Download PowerPoint

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

• Sampling methods
• Simple random sampling
• Stratified sampling
• Cluster sampling
• Likert scales
• Reproducibility

 Statistics

• Null hypothesis
• Statistical power
• Probability distribution
• Effect size
• Poisson distribution

Research bias

• Optimism bias
• Cognitive bias
• Implicit bias
• Hawthorne effect
• Anchoring bias
• Explicit bias

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

There are several reasons to conduct a literature review at the beginning of a research project:

• To familiarize yourself with the current state of knowledge on your topic
• To ensure that you’re not just repeating what others have already done
• To identify gaps in knowledge and unresolved problems that your research can address
• To develop your theoretical framework and methodology
• To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

The literature review usually comes near the beginning of your thesis or dissertation . After the introduction , it grounds your research in a scholarly field and leads directly to your theoretical framework or methodology .

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other  academic texts , with an introduction , a main body, and a conclusion .

An  annotated bibliography is a list of  source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a  paper .  

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, September 11). How to Write a Literature Review | Guide, Examples, & Templates. Scribbr. Retrieved November 14, 2023, from https://www.scribbr.com/dissertation/literature-review/

Is this article helpful?

Shona McCombes

Other students also liked, what is a theoretical framework | guide to organizing, what is a research methodology | steps & tips, how to write a research proposal | examples & templates, what is your plagiarism score.

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Writing a Literature Review

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

A literature review is a document or section of a document that collects key sources on a topic and discusses those sources in conversation with each other (also called synthesis ). The lit review is an important genre in many disciplines, not just literature (i.e., the study of works of literature such as novels and plays). When we say “literature review” or refer to “the literature,” we are talking about the research ( scholarship ) in a given field. You will often see the terms “the research,” “the scholarship,” and “the literature” used mostly interchangeably.

Where, when, and why would I write a lit review?

There are a number of different situations where you might write a literature review, each with slightly different expectations; different disciplines, too, have field-specific expectations for what a literature review is and does. For instance, in the humanities, authors might include more overt argumentation and interpretation of source material in their literature reviews, whereas in the sciences, authors are more likely to report study designs and results in their literature reviews; these differences reflect these disciplines’ purposes and conventions in scholarship. You should always look at examples from your own discipline and talk to professors or mentors in your field to be sure you understand your discipline’s conventions, for literature reviews as well as for any other genre.

A literature review can be a part of a research paper or scholarly article, usually falling after the introduction and before the research methods sections. In these cases, the lit review just needs to cover scholarship that is important to the issue you are writing about; sometimes it will also cover key sources that informed your research methodology.

Lit reviews can also be standalone pieces, either as assignments in a class or as publications. In a class, a lit review may be assigned to help students familiarize themselves with a topic and with scholarship in their field, get an idea of the other researchers working on the topic they’re interested in, find gaps in existing research in order to propose new projects, and/or develop a theoretical framework and methodology for later research. As a publication, a lit review usually is meant to help make other scholars’ lives easier by collecting and summarizing, synthesizing, and analyzing existing research on a topic. This can be especially helpful for students or scholars getting into a new research area, or for directing an entire community of scholars toward questions that have not yet been answered.

What are the parts of a lit review?

Most lit reviews use a basic introduction-body-conclusion structure; if your lit review is part of a larger paper, the introduction and conclusion pieces may be just a few sentences while you focus most of your attention on the body. If your lit review is a standalone piece, the introduction and conclusion take up more space and give you a place to discuss your goals, research methods, and conclusions separately from where you discuss the literature itself.

Introduction:

• An introductory paragraph that explains what your working topic and thesis is
• A forecast of key topics or texts that will appear in the review
• Potentially, a description of how you found sources and how you analyzed them for inclusion and discussion in the review (more often found in published, standalone literature reviews than in lit review sections in an article or research paper)
• Summarize and synthesize: Give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: Don’t just paraphrase other researchers – add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically Evaluate: Mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: Use transition words and topic sentence to draw connections, comparisons, and contrasts.

Conclusion:

• Summarize the key findings you have taken from the literature and emphasize their significance
• Connect it back to your primary research question

How should I organize my lit review?

Lit reviews can take many different organizational patterns depending on what you are trying to accomplish with the review. Here are some examples:

• Chronological : The simplest approach is to trace the development of the topic over time, which helps familiarize the audience with the topic (for instance if you are introducing something that is not commonly known in your field). If you choose this strategy, be careful to avoid simply listing and summarizing sources in order. Try to analyze the patterns, turning points, and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred (as mentioned previously, this may not be appropriate in your discipline — check with a teacher or mentor if you’re unsure).
• Thematic : If you have found some recurring central themes that you will continue working with throughout your piece, you can organize your literature review into subsections that address different aspects of the topic. For example, if you are reviewing literature about women and religion, key themes can include the role of women in churches and the religious attitude towards women.
• Qualitative versus quantitative research
• Empirical versus theoretical scholarship
• Divide the research by sociological, historical, or cultural sources
• Theoretical : In many humanities articles, the literature review is the foundation for the theoretical framework. You can use it to discuss various theories, models, and definitions of key concepts. You can argue for the relevance of a specific theoretical approach or combine various theorical concepts to create a framework for your research.

What are some strategies or tips I can use while writing my lit review?

Any lit review is only as good as the research it discusses; make sure your sources are well-chosen and your research is thorough. Don’t be afraid to do more research if you discover a new thread as you’re writing. More info on the research process is available in our "Conducting Research" resources .

As you’re doing your research, create an annotated bibliography ( see our page on the this type of document ). Much of the information used in an annotated bibliography can be used also in a literature review, so you’ll be not only partially drafting your lit review as you research, but also developing your sense of the larger conversation going on among scholars, professionals, and any other stakeholders in your topic.

Usually you will need to synthesize research rather than just summarizing it. This means drawing connections between sources to create a picture of the scholarly conversation on a topic over time. Many student writers struggle to synthesize because they feel they don’t have anything to add to the scholars they are citing; here are some strategies to help you:

• It often helps to remember that the point of these kinds of syntheses is to show your readers how you understand your research, to help them read the rest of your paper.
• Writing teachers often say synthesis is like hosting a dinner party: imagine all your sources are together in a room, discussing your topic. What are they saying to each other?
• Look at the in-text citations in each paragraph. Are you citing just one source for each paragraph? This usually indicates summary only. When you have multiple sources cited in a paragraph, you are more likely to be synthesizing them (not always, but often
• Read more about synthesis here.

The most interesting literature reviews are often written as arguments (again, as mentioned at the beginning of the page, this is discipline-specific and doesn’t work for all situations). Often, the literature review is where you can establish your research as filling a particular gap or as relevant in a particular way. You have some chance to do this in your introduction in an article, but the literature review section gives a more extended opportunity to establish the conversation in the way you would like your readers to see it. You can choose the intellectual lineage you would like to be part of and whose definitions matter most to your thinking (mostly humanities-specific, but this goes for sciences as well). In addressing these points, you argue for your place in the conversation, which tends to make the lit review more compelling than a simple reporting of other sources.

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

The Literature Review

29 The Five ‘C’s of Writing a Literature Review

To help you frame and write your literature review, think about these five ‘c’s (Callahan, 2014).

• Cite the material you have referred to and used to help you define the research problem that you will study.
• Compare the various arguments, theories, methods, and findings expressed in the literature.  For example, describe where the various researchers agree and where they disagree. Describe the similarities and dissimilarities in approaches to studying related research problems.
• Contrast the various arguments, themes, methods, approaches, and controversies apparent and/or described in the literature.  For example, describe what major areas are contested, controversial and/or still in debate.
• Critique the literature.  Describe which arguments you find more persuasive and explain why.  Explain which approaches, findings, and methods seem most reliable, valid, appropriate, and/or most popular and why.  Pay attention to the verbs you use to describe what previous researchers have stated (e.g. asserts, demonstrates, argues, clarifies, etc.).
• Connect the various research studies you reviewed.  Describe how your work utilizes, draws upon, departs from, synthesizes, adds to or extends previous research studies.

An Introduction to Research Methods in Sociology by Valerie A. Sheppard is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Share This Book

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

To help you frame and write your literature review, think about these five c’s (Callahan, 2014):

• Cite the material you have referred to and used to help you define the research problem that you will study.
• Compare the various arguments, theories, methods, and findings expressed in the literature.For example, describe where the various researchers agree and where they disagree. Describe the similarities and dissimilarities in approaches to studying related research problems.
• Contrast the various arguments, themes, methods, approaches, and controversies apparent and/or described in the literature. For example, describe what major areas are contested, controversial and/or still in debate.
• Critique the literature. Describe which arguments you find more persuasive and explain why. Explain which approaches, findings, and methods seem most reliable, valid, appropriate, and/or most popular and why. Pay attention to the verbs you use to describe what previous researchers have stated (e.g., asserts, demonstrates, argues, clarifies, etc.).
• Connect the various research studies you reviewed. Describe how your work utilizes, draws upon, departs from, synthesizes, adds to or extends previous research studies.

Research Methods, Data Collection and Ethics Copyright © 2020 by Valerie Sheppard is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Share This Book

• USC Libraries
• Research Guides

Organizing Your Social Sciences Research Paper

• 5. The Literature Review
• Purpose of Guide
• Design Flaws to Avoid
• Independent and Dependent Variables
• Glossary of Research Terms
• Reading Research Effectively
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Choosing a Title
• Making an Outline
• Paragraph Development
• Research Process Video Series
• Executive Summary
• The C.A.R.S. Model
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tiertiary Sources
• Scholarly vs. Popular Publications
• Qualitative Methods
• Quantitative Methods
• Insiderness
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Writing Concisely
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Generative AI and Writing
• USC Libraries Tutorials and Other Guides
• Bibliography

A literature review surveys prior research published in books, scholarly articles, and any other sources relevant to a particular issue, area of research, or theory, and by so doing, provides a description, summary, and critical evaluation of these works in relation to the research problem being investigated. Literature reviews are designed to provide an overview of sources you have used in researching a particular topic and to demonstrate to your readers how your research fits within existing scholarship about the topic.

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . Fourth edition. Thousand Oaks, CA: SAGE, 2014.

Importance of a Good Literature Review

A literature review may consist of simply a summary of key sources, but in the social sciences, a literature review usually has an organizational pattern and combines both summary and synthesis, often within specific conceptual categories . A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information in a way that informs how you are planning to investigate a research problem. The analytical features of a literature review might:

• Give a new interpretation of old material or combine new with old interpretations,
• Trace the intellectual progression of the field, including major debates,
• Depending on the situation, evaluate the sources and advise the reader on the most pertinent or relevant research, or
• Usually in the conclusion of a literature review, identify where gaps exist in how a problem has been researched to date.

Given this, the purpose of a literature review is to:

• Place each work in the context of its contribution to understanding the research problem being studied.
• Describe the relationship of each work to the others under consideration.
• Identify new ways to interpret prior research.
• Reveal any gaps that exist in the literature.
• Resolve conflicts amongst seemingly contradictory previous studies.
• Identify areas of prior scholarship to prevent duplication of effort.
• Point the way in fulfilling a need for additional research.
• Locate your own research within the context of existing literature [very important].

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper. 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . Los Angeles, CA: SAGE, 2011; Knopf, Jeffrey W. "Doing a Literature Review." PS: Political Science and Politics 39 (January 2006): 127-132; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012.

Types of Literature Reviews

It is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the primary studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally among scholars that become part of the body of epistemological traditions within the field.

In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews. Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are a number of approaches you could adopt depending upon the type of analysis underpinning your study.

Argumentative Review This form examines literature selectively in order to support or refute an argument, deeply embedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to make summary claims of the sort found in systematic reviews [see below].

Integrative Review Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses or research problems. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication. This is the most common form of review in the social sciences.

Historical Review Few things rest in isolation from historical precedent. Historical literature reviews focus on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review A review does not always focus on what someone said [findings], but how they came about saying what they say [method of analysis]. Reviewing methods of analysis provides a framework of understanding at different levels [i.e. those of theory, substantive fields, research approaches, and data collection and analysis techniques], how researchers draw upon a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection, and data analysis. This approach helps highlight ethical issues which you should be aware of and consider as you go through your own study.

Systematic Review This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyze data from the studies that are included in the review. The goal is to deliberately document, critically evaluate, and summarize scientifically all of the research about a clearly defined research problem . Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?" This type of literature review is primarily applied to examining prior research studies in clinical medicine and allied health fields, but it is increasingly being used in the social sciences.

Theoretical Review The purpose of this form is to examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review helps to establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

NOTE : Most often the literature review will incorporate some combination of types. For example, a review that examines literature supporting or refuting an argument, assumption, or philosophical problem related to the research problem will also need to include writing supported by sources that establish the history of these arguments in the literature.

Baumeister, Roy F. and Mark R. Leary. "Writing Narrative Literature Reviews."  Review of General Psychology 1 (September 1997): 311-320; Mark R. Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Kennedy, Mary M. "Defining a Literature." Educational Researcher 36 (April 2007): 139-147; Petticrew, Mark and Helen Roberts. Systematic Reviews in the Social Sciences: A Practical Guide . Malden, MA: Blackwell Publishers, 2006; Torracro, Richard. "Writing Integrative Literature Reviews: Guidelines and Examples." Human Resource Development Review 4 (September 2005): 356-367; Rocco, Tonette S. and Maria S. Plakhotnik. "Literature Reviews, Conceptual Frameworks, and Theoretical Frameworks: Terms, Functions, and Distinctions." Human Ressource Development Review 8 (March 2008): 120-130; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

Structure and Writing Style

I.  Thinking About Your Literature Review

The structure of a literature review should include the following in support of understanding the research problem :

• An overview of the subject, issue, or theory under consideration, along with the objectives of the literature review,
• Division of works under review into themes or categories [e.g. works that support a particular position, those against, and those offering alternative approaches entirely],
• An explanation of how each work is similar to and how it varies from the others,
• Conclusions as to which pieces are best considered in their argument, are most convincing of their opinions, and make the greatest contribution to the understanding and development of their area of research.

The critical evaluation of each work should consider :

• Provenance -- what are the author's credentials? Are the author's arguments supported by evidence [e.g. primary historical material, case studies, narratives, statistics, recent scientific findings]?
• Methodology -- were the techniques used to identify, gather, and analyze the data appropriate to addressing the research problem? Was the sample size appropriate? Were the results effectively interpreted and reported?
• Objectivity -- is the author's perspective even-handed or prejudicial? Is contrary data considered or is certain pertinent information ignored to prove the author's point?
• Persuasiveness -- which of the author's theses are most convincing or least convincing?
• Validity -- are the author's arguments and conclusions convincing? Does the work ultimately contribute in any significant way to an understanding of the subject?

II.  Development of the Literature Review

Four Basic Stages of Writing 1.  Problem formulation -- which topic or field is being examined and what are its component issues? 2.  Literature search -- finding materials relevant to the subject being explored. 3.  Data evaluation -- determining which literature makes a significant contribution to the understanding of the topic. 4.  Analysis and interpretation -- discussing the findings and conclusions of pertinent literature.

Consider the following issues before writing the literature review: Clarify If your assignment is not specific about what form your literature review should take, seek clarification from your professor by asking these questions: 1.  Roughly how many sources would be appropriate to include? 2.  What types of sources should I review (books, journal articles, websites; scholarly versus popular sources)? 3.  Should I summarize, synthesize, or critique sources by discussing a common theme or issue? 4.  Should I evaluate the sources in any way beyond evaluating how they relate to understanding the research problem? 5.  Should I provide subheadings and other background information, such as definitions and/or a history? Find Models Use the exercise of reviewing the literature to examine how authors in your discipline or area of interest have composed their literature review sections. Read them to get a sense of the types of themes you might want to look for in your own research or to identify ways to organize your final review. The bibliography or reference section of sources you've already read, such as required readings in the course syllabus, are also excellent entry points into your own research. Narrow the Topic The narrower your topic, the easier it will be to limit the number of sources you need to read in order to obtain a good survey of relevant resources. Your professor will probably not expect you to read everything that's available about the topic, but you'll make the act of reviewing easier if you first limit scope of the research problem. A good strategy is to begin by searching the USC Libraries Catalog for recent books about the topic and review the table of contents for chapters that focuses on specific issues. You can also review the indexes of books to find references to specific issues that can serve as the focus of your research. For example, a book surveying the history of the Israeli-Palestinian conflict may include a chapter on the role Egypt has played in mediating the conflict, or look in the index for the pages where Egypt is mentioned in the text. Consider Whether Your Sources are Current Some disciplines require that you use information that is as current as possible. This is particularly true in disciplines in medicine and the sciences where research conducted becomes obsolete very quickly as new discoveries are made. However, when writing a review in the social sciences, a survey of the history of the literature may be required. In other words, a complete understanding the research problem requires you to deliberately examine how knowledge and perspectives have changed over time. Sort through other current bibliographies or literature reviews in the field to get a sense of what your discipline expects. You can also use this method to explore what is considered by scholars to be a "hot topic" and what is not.

III.  Ways to Organize Your Literature Review

Chronology of Events If your review follows the chronological method, you could write about the materials according to when they were published. This approach should only be followed if a clear path of research building on previous research can be identified and that these trends follow a clear chronological order of development. For example, a literature review that focuses on continuing research about the emergence of German economic power after the fall of the Soviet Union. By Publication Order your sources by publication chronology, then, only if the order demonstrates a more important trend. For instance, you could order a review of literature on environmental studies of brown fields if the progression revealed, for example, a change in the soil collection practices of the researchers who wrote and/or conducted the studies. Thematic [“conceptual categories”] A thematic literature review is the most common approach to summarizing prior research in the social and behavioral sciences. Thematic reviews are organized around a topic or issue, rather than the progression of time, although the progression of time may still be incorporated into a thematic review. For example, a review of the Internet’s impact on American presidential politics could focus on the development of online political satire. While the study focuses on one topic, the Internet’s impact on American presidential politics, it would still be organized chronologically reflecting technological developments in media. The difference in this example between a "chronological" and a "thematic" approach is what is emphasized the most: themes related to the role of the Internet in presidential politics. Note that more authentic thematic reviews tend to break away from chronological order. A review organized in this manner would shift between time periods within each section according to the point being made. Methodological A methodological approach focuses on the methods utilized by the researcher. For the Internet in American presidential politics project, one methodological approach would be to look at cultural differences between the portrayal of American presidents on American, British, and French websites. Or the review might focus on the fundraising impact of the Internet on a particular political party. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed.

Other Sections of Your Literature Review Once you've decided on the organizational method for your literature review, the sections you need to include in the paper should be easy to figure out because they arise from your organizational strategy. In other words, a chronological review would have subsections for each vital time period; a thematic review would have subtopics based upon factors that relate to the theme or issue. However, sometimes you may need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. However, only include what is necessary for the reader to locate your study within the larger scholarship about the research problem.

Here are examples of other sections, usually in the form of a single paragraph, you may need to include depending on the type of review you write:

• Current Situation : Information necessary to understand the current topic or focus of the literature review.
• Sources Used : Describes the methods and resources [e.g., databases] you used to identify the literature you reviewed.
• History : The chronological progression of the field, the research literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.
• Selection Methods : Criteria you used to select (and perhaps exclude) sources in your literature review. For instance, you might explain that your review includes only peer-reviewed [i.e., scholarly] sources.
• Standards : Description of the way in which you present your information.
• Questions for Further Research : What questions about the field has the review sparked? How will you further your research as a result of the review?

IV.  Writing Your Literature Review

Once you've settled on how to organize your literature review, you're ready to write each section. When writing your review, keep in mind these issues.

Use Evidence A literature review section is, in this sense, just like any other academic research paper. Your interpretation of the available sources must be backed up with evidence [citations] that demonstrates that what you are saying is valid. Be Selective Select only the most important points in each source to highlight in the review. The type of information you choose to mention should relate directly to the research problem, whether it is thematic, methodological, or chronological. Related items that provide additional information, but that are not key to understanding the research problem, can be included in a list of further readings . Use Quotes Sparingly Some short quotes are appropriate if you want to emphasize a point, or if what an author stated cannot be easily paraphrased. Sometimes you may need to quote certain terminology that was coined by the author, is not common knowledge, or taken directly from the study. Do not use extensive quotes as a substitute for using your own words in reviewing the literature. Summarize and Synthesize Remember to summarize and synthesize your sources within each thematic paragraph as well as throughout the review. Recapitulate important features of a research study, but then synthesize it by rephrasing the study's significance and relating it to your own work and the work of others. Keep Your Own Voice While the literature review presents others' ideas, your voice [the writer's] should remain front and center. For example, weave references to other sources into what you are writing but maintain your own voice by starting and ending the paragraph with your own ideas and wording. Use Caution When Paraphrasing When paraphrasing a source that is not your own, be sure to represent the author's information or opinions accurately and in your own words. Even when paraphrasing an author’s work, you still must provide a citation to that work.

V.  Common Mistakes to Avoid

These are the most common mistakes made in reviewing social science research literature.

• Sources in your literature review do not clearly relate to the research problem;
• You do not take sufficient time to define and identify the most relevant sources to use in the literature review related to the research problem;
• Relies exclusively on secondary analytical sources rather than including relevant primary research studies or data;
• Uncritically accepts another researcher's findings and interpretations as valid, rather than examining critically all aspects of the research design and analysis;
• Does not describe the search procedures that were used in identifying the literature to review;
• Reports isolated statistical results rather than synthesizing them in chi-squared or meta-analytic methods; and,
• Only includes research that validates assumptions and does not consider contrary findings and alternative interpretations found in the literature.

Cook, Kathleen E. and Elise Murowchick. “Do Literature Review Skills Transfer from One Course to Another?” Psychology Learning and Teaching 13 (March 2014): 3-11; Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . London: SAGE, 2011; Literature Review Handout. Online Writing Center. Liberty University; Literature Reviews. The Writing Center. University of North Carolina; Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: SAGE, 2016; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012; Randolph, Justus J. “A Guide to Writing the Dissertation Literature Review." Practical Assessment, Research, and Evaluation. vol. 14, June 2009; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016; Taylor, Dena. The Literature Review: A Few Tips On Conducting It. University College Writing Centre. University of Toronto; Writing a Literature Review. Academic Skills Centre. University of Canberra.

Writing Tip

Break Out of Your Disciplinary Box!

Thinking interdisciplinarily about a research problem can be a rewarding exercise in applying new ideas, theories, or concepts to an old problem. For example, what might cultural anthropologists say about the continuing conflict in the Middle East? In what ways might geographers view the need for better distribution of social service agencies in large cities than how social workers might study the issue? You don’t want to substitute a thorough review of core research literature in your discipline for studies conducted in other fields of study. However, particularly in the social sciences, thinking about research problems from multiple vectors is a key strategy for finding new solutions to a problem or gaining a new perspective. Consult with a librarian about identifying research databases in other disciplines; almost every field of study has at least one comprehensive database devoted to indexing its research literature.

Frodeman, Robert. The Oxford Handbook of Interdisciplinarity . New York: Oxford University Press, 2010.

Another Writing Tip

Don't Just Review for Content!

While conducting a review of the literature, maximize the time you devote to writing this part of your paper by thinking broadly about what you should be looking for and evaluating. Review not just what scholars are saying, but how are they saying it. Some questions to ask:

• How are they organizing their ideas?
• What methods have they used to study the problem?
• What theories have been used to explain, predict, or understand their research problem?
• What sources have they cited to support their conclusions?
• How have they used non-textual elements [e.g., charts, graphs, figures, etc.] to illustrate key points?

When you begin to write your literature review section, you'll be glad you dug deeper into how the research was designed and constructed because it establishes a means for developing more substantial analysis and interpretation of the research problem.

Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1 998.

Yet Another Writing Tip

When Do I Know I Can Stop Looking and Move On?

Here are several strategies you can utilize to assess whether you've thoroughly reviewed the literature:

• Look for repeating patterns in the research findings . If the same thing is being said, just by different people, then this likely demonstrates that the research problem has hit a conceptual dead end. At this point consider: Does your study extend current research?  Does it forge a new path? Or, does is merely add more of the same thing being said?
• Look at sources the authors cite to in their work . If you begin to see the same researchers cited again and again, then this is often an indication that no new ideas have been generated to address the research problem.
• Search Google Scholar to identify who has subsequently cited leading scholars already identified in your literature review [see next sub-tab]. This is called citation tracking and there are a number of sources that can help you identify who has cited whom, particularly scholars from outside of your discipline. Here again, if the same authors are being cited again and again, this may indicate no new literature has been written on the topic.

Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: Sage, 2016; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

• << Previous: Theoretical Framework
• Next: Citation Tracking >>
• Last Updated: Oct 10, 2023 1:30 PM
• URL: https://libguides.usc.edu/writingguide

• UWF Libraries

Literature Review: Conducting & Writing

• Sample Literature Reviews
• Steps for Conducting a Lit Review
• Finding "The Literature"
• Organizing/Writing
• Chicago: Notes Bibliography

Sample Lit Reviews from Communication Arts

Have an exemplary literature review.

• Literature Review Sample 1
• Literature Review Sample 2
• Literature Review Sample 3

Have you written a stellar literature review you care to share for teaching purposes?

Are you an instructor who has received an exemplary literature review and have permission from the student to post?

Please contact Britt McGowan at [email protected] for inclusion in this guide. All disciplines welcome and encouraged.

• << Previous: MLA Style
• Next: Get Help! >>
• Last Updated: Aug 24, 2023 9:59 AM
• URL: https://libguides.uwf.edu/litreview

Harvey Cushing/John Hay Whitney Medical Library

• Collections
• Research Help

YSN Doctoral Programs: Steps in Conducting a Literature Review

• Biomedical Databases
• Global (Public Health) Databases
• Soc. Sci., History, and Law Databases
• Grey Literature
• Trials Registers
• Data and Statistics
• Public Policy
• Google Tips
• Recommended Books
• Steps in Conducting a Literature Review

What is a literature review?

A literature review is an integrated analysis -- not just a summary-- of scholarly writings and other relevant evidence related directly to your research question.  That is, it represents a synthesis of the evidence that provides background information on your topic and shows a association between the evidence and your research question.

A literature review may be a stand alone work or the introduction to a larger research paper, depending on the assignment.  Rely heavily on the guidelines your instructor has given you.

Why is it important?

A literature review is important because it:

• Explains the background of research on a topic.
• Demonstrates why a topic is significant to a subject area.
• Discovers relationships between research studies/ideas.
• Identifies major themes, concepts, and researchers on a topic.
• Identifies critical gaps and points of disagreement.
• Discusses further research questions that logically come out of the previous studies.

APA7 Style resources

APA Style Blog - for those harder to find answers

1. Choose a topic. Define your research question.

Your literature review should be guided by your central research question.  The literature represents background and research developments related to a specific research question, interpreted and analyzed by you in a synthesized way.

• Make sure your research question is not too broad or too narrow.  Is it manageable?
• Begin writing down terms that are related to your question. These will be useful for searches later.
• If you have the opportunity, discuss your topic with your professor and your class mates.

2. Decide on the scope of your review

How many studies do you need to look at? How comprehensive should it be? How many years should it cover? 

• This may depend on your assignment.  How many sources does the assignment require?

3. Select the databases you will use to conduct your searches.

Make a list of the databases you will search. 

Where to find databases:

• use the tabs on this guide
• Find other databases in the Nursing Information Resources web page
• More on the Medical Library web page
• ... and more on the Yale University Library web page

4. Conduct your searches to find the evidence. Keep track of your searches.

• Use the key words in your question, as well as synonyms for those words, as terms in your search. Use the database tutorials for help.
• Save the searches in the databases. This saves time when you want to redo, or modify, the searches. It is also helpful to use as a guide is the searches are not finding any useful results.
• Review the abstracts of research studies carefully. This will save you time.
• Use the bibliographies and references of research studies you find to locate others.
• Check with your professor, or a subject expert in the field, if you are missing any key works in the field.
• Ask your librarian for help at any time.
• Use a citation manager, such as EndNote as the repository for your citations. See the EndNote tutorials for help.

Review the literature

Some questions to help you analyze the research:

• What was the research question of the study you are reviewing? What were the authors trying to discover?
• Was the research funded by a source that could influence the findings?
• What were the research methodologies? Analyze its literature review, the samples and variables used, the results, and the conclusions.
• Does the research seem to be complete? Could it have been conducted more soundly? What further questions does it raise?
• If there are conflicting studies, why do you think that is?
• How are the authors viewed in the field? Has this study been cited? If so, how has it been analyzed?

Tips: 

• Review the abstracts carefully.  
• Keep careful notes so that you may track your thought processes during the research process.
• Create a matrix of the studies for easy analysis, and synthesis, across all of the studies.
• << Previous: Recommended Books
• Last Updated: Oct 31, 2023 3:00 PM
• URL: https://guides.library.yale.edu/YSNDoctoral

Literature Reviews

What this handout is about.

This handout will explain what literature reviews are and offer insights into the form and construction of literature reviews in the humanities, social sciences, and sciences.

Introduction

OK. You’ve got to write a literature review. You dust off a novel and a book of poetry, settle down in your chair, and get ready to issue a “thumbs up” or “thumbs down” as you leaf through the pages. “Literature review” done. Right?

Wrong! The “literature” of a literature review refers to any collection of materials on a topic, not necessarily the great literary texts of the world. “Literature” could be anything from a set of government pamphlets on British colonial methods in Africa to scholarly articles on the treatment of a torn ACL. And a review does not necessarily mean that your reader wants you to give your personal opinion on whether or not you liked these sources.

What is a literature review, then?

A literature review discusses published information in a particular subject area, and sometimes information in a particular subject area within a certain time period.

A literature review can be just a simple summary of the sources, but it usually has an organizational pattern and combines both summary and synthesis. A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information. It might give a new interpretation of old material or combine new with old interpretations. Or it might trace the intellectual progression of the field, including major debates. And depending on the situation, the literature review may evaluate the sources and advise the reader on the most pertinent or relevant.

But how is a literature review different from an academic research paper?

The main focus of an academic research paper is to develop a new argument, and a research paper is likely to contain a literature review as one of its parts. In a research paper, you use the literature as a foundation and as support for a new insight that you contribute. The focus of a literature review, however, is to summarize and synthesize the arguments and ideas of others without adding new contributions.

Why do we write literature reviews?

Literature reviews provide you with a handy guide to a particular topic. If you have limited time to conduct research, literature reviews can give you an overview or act as a stepping stone. For professionals, they are useful reports that keep them up to date with what is current in the field. For scholars, the depth and breadth of the literature review emphasizes the credibility of the writer in his or her field. Literature reviews also provide a solid background for a research paper’s investigation. Comprehensive knowledge of the literature of the field is essential to most research papers.

Who writes these things, anyway?

Literature reviews are written occasionally in the humanities, but mostly in the sciences and social sciences; in experiment and lab reports, they constitute a section of the paper. Sometimes a literature review is written as a paper in itself.

Let’s get to it! What should I do before writing the literature review?

If your assignment is not very specific, seek clarification from your instructor:

• Roughly how many sources should you include?
• What types of sources (books, journal articles, websites)?
• Should you summarize, synthesize, or critique your sources by discussing a common theme or issue?
• Should you evaluate your sources?
• Should you provide subheadings and other background information, such as definitions and/or a history?

Find models

Look for other literature reviews in your area of interest or in the discipline and read them to get a sense of the types of themes you might want to look for in your own research or ways to organize your final review. You can simply put the word “review” in your search engine along with your other topic terms to find articles of this type on the Internet or in an electronic database. The bibliography or reference section of sources you’ve already read are also excellent entry points into your own research.

Narrow your topic

There are hundreds or even thousands of articles and books on most areas of study. The narrower your topic, the easier it will be to limit the number of sources you need to read in order to get a good survey of the material. Your instructor will probably not expect you to read everything that’s out there on the topic, but you’ll make your job easier if you first limit your scope.

Keep in mind that UNC Libraries have research guides and to databases relevant to many fields of study. You can reach out to the subject librarian for a consultation: https://library.unc.edu/support/consultations/ .

And don’t forget to tap into your professor’s (or other professors’) knowledge in the field. Ask your professor questions such as: “If you had to read only one book from the 90’s on topic X, what would it be?” Questions such as this help you to find and determine quickly the most seminal pieces in the field.

Consider whether your sources are current

Some disciplines require that you use information that is as current as possible. In the sciences, for instance, treatments for medical problems are constantly changing according to the latest studies. Information even two years old could be obsolete. However, if you are writing a review in the humanities, history, or social sciences, a survey of the history of the literature may be what is needed, because what is important is how perspectives have changed through the years or within a certain time period. Try sorting through some other current bibliographies or literature reviews in the field to get a sense of what your discipline expects. You can also use this method to consider what is currently of interest to scholars in this field and what is not.

Strategies for writing the literature review

Find a focus.

A literature review, like a term paper, is usually organized around ideas, not the sources themselves as an annotated bibliography would be organized. This means that you will not just simply list your sources and go into detail about each one of them, one at a time. No. As you read widely but selectively in your topic area, consider instead what themes or issues connect your sources together. Do they present one or different solutions? Is there an aspect of the field that is missing? How well do they present the material and do they portray it according to an appropriate theory? Do they reveal a trend in the field? A raging debate? Pick one of these themes to focus the organization of your review.

Convey it to your reader

A literature review may not have a traditional thesis statement (one that makes an argument), but you do need to tell readers what to expect. Try writing a simple statement that lets the reader know what is your main organizing principle. Here are a couple of examples:

The current trend in treatment for congestive heart failure combines surgery and medicine. More and more cultural studies scholars are accepting popular media as a subject worthy of academic consideration.

Consider organization

You’ve got a focus, and you’ve stated it clearly and directly. Now what is the most effective way of presenting the information? What are the most important topics, subtopics, etc., that your review needs to include? And in what order should you present them? Develop an organization for your review at both a global and local level:

First, cover the basic categories

Just like most academic papers, literature reviews also must contain at least three basic elements: an introduction or background information section; the body of the review containing the discussion of sources; and, finally, a conclusion and/or recommendations section to end the paper. The following provides a brief description of the content of each:

• Introduction: Gives a quick idea of the topic of the literature review, such as the central theme or organizational pattern.
• Body: Contains your discussion of sources and is organized either chronologically, thematically, or methodologically (see below for more information on each).
• Conclusions/Recommendations: Discuss what you have drawn from reviewing literature so far. Where might the discussion proceed?

Organizing the body

Once you have the basic categories in place, then you must consider how you will present the sources themselves within the body of your paper. Create an organizational method to focus this section even further.

To help you come up with an overall organizational framework for your review, consider the following scenario:

You’ve decided to focus your literature review on materials dealing with sperm whales. This is because you’ve just finished reading Moby Dick, and you wonder if that whale’s portrayal is really real. You start with some articles about the physiology of sperm whales in biology journals written in the 1980’s. But these articles refer to some British biological studies performed on whales in the early 18th century. So you check those out. Then you look up a book written in 1968 with information on how sperm whales have been portrayed in other forms of art, such as in Alaskan poetry, in French painting, or on whale bone, as the whale hunters in the late 19th century used to do. This makes you wonder about American whaling methods during the time portrayed in Moby Dick, so you find some academic articles published in the last five years on how accurately Herman Melville portrayed the whaling scene in his novel.

Now consider some typical ways of organizing the sources into a review:

• Chronological: If your review follows the chronological method, you could write about the materials above according to when they were published. For instance, first you would talk about the British biological studies of the 18th century, then about Moby Dick, published in 1851, then the book on sperm whales in other art (1968), and finally the biology articles (1980s) and the recent articles on American whaling of the 19th century. But there is relatively no continuity among subjects here. And notice that even though the sources on sperm whales in other art and on American whaling are written recently, they are about other subjects/objects that were created much earlier. Thus, the review loses its chronological focus.
• By publication: Order your sources by publication chronology, then, only if the order demonstrates a more important trend. For instance, you could order a review of literature on biological studies of sperm whales if the progression revealed a change in dissection practices of the researchers who wrote and/or conducted the studies.
• By trend: A better way to organize the above sources chronologically is to examine the sources under another trend, such as the history of whaling. Then your review would have subsections according to eras within this period. For instance, the review might examine whaling from pre-1600-1699, 1700-1799, and 1800-1899. Under this method, you would combine the recent studies on American whaling in the 19th century with Moby Dick itself in the 1800-1899 category, even though the authors wrote a century apart.
• Thematic: Thematic reviews of literature are organized around a topic or issue, rather than the progression of time. However, progression of time may still be an important factor in a thematic review. For instance, the sperm whale review could focus on the development of the harpoon for whale hunting. While the study focuses on one topic, harpoon technology, it will still be organized chronologically. The only difference here between a “chronological” and a “thematic” approach is what is emphasized the most: the development of the harpoon or the harpoon technology.But more authentic thematic reviews tend to break away from chronological order. For instance, a thematic review of material on sperm whales might examine how they are portrayed as “evil” in cultural documents. The subsections might include how they are personified, how their proportions are exaggerated, and their behaviors misunderstood. A review organized in this manner would shift between time periods within each section according to the point made.
• Methodological: A methodological approach differs from the two above in that the focusing factor usually does not have to do with the content of the material. Instead, it focuses on the “methods” of the researcher or writer. For the sperm whale project, one methodological approach would be to look at cultural differences between the portrayal of whales in American, British, and French art work. Or the review might focus on the economic impact of whaling on a community. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed. Once you’ve decided on the organizational method for the body of the review, the sections you need to include in the paper should be easy to figure out. They should arise out of your organizational strategy. In other words, a chronological review would have subsections for each vital time period. A thematic review would have subtopics based upon factors that relate to the theme or issue.

Sometimes, though, you might need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. Put in only what is necessary. Here are a few other sections you might want to consider:

• Current Situation: Information necessary to understand the topic or focus of the literature review.
• History: The chronological progression of the field, the literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.
• Methods and/or Standards: The criteria you used to select the sources in your literature review or the way in which you present your information. For instance, you might explain that your review includes only peer-reviewed articles and journals.

Questions for Further Research: What questions about the field has the review sparked? How will you further your research as a result of the review?

Begin composing

Once you’ve settled on a general pattern of organization, you’re ready to write each section. There are a few guidelines you should follow during the writing stage as well. Here is a sample paragraph from a literature review about sexism and language to illuminate the following discussion:

However, other studies have shown that even gender-neutral antecedents are more likely to produce masculine images than feminine ones (Gastil, 1990). Hamilton (1988) asked students to complete sentences that required them to fill in pronouns that agreed with gender-neutral antecedents such as “writer,” “pedestrian,” and “persons.” The students were asked to describe any image they had when writing the sentence. Hamilton found that people imagined 3.3 men to each woman in the masculine “generic” condition and 1.5 men per woman in the unbiased condition. Thus, while ambient sexism accounted for some of the masculine bias, sexist language amplified the effect. (Source: Erika Falk and Jordan Mills, “Why Sexist Language Affects Persuasion: The Role of Homophily, Intended Audience, and Offense,” Women and Language19:2).

Use evidence

In the example above, the writers refer to several other sources when making their point. A literature review in this sense is just like any other academic research paper. Your interpretation of the available sources must be backed up with evidence to show that what you are saying is valid.

Be selective

Select only the most important points in each source to highlight in the review. The type of information you choose to mention should relate directly to the review’s focus, whether it is thematic, methodological, or chronological.

Use quotes sparingly

Falk and Mills do not use any direct quotes. That is because the survey nature of the literature review does not allow for in-depth discussion or detailed quotes from the text. Some short quotes here and there are okay, though, if you want to emphasize a point, or if what the author said just cannot be rewritten in your own words. Notice that Falk and Mills do quote certain terms that were coined by the author, not common knowledge, or taken directly from the study. But if you find yourself wanting to put in more quotes, check with your instructor.

Summarize and synthesize

Remember to summarize and synthesize your sources within each paragraph as well as throughout the review. The authors here recapitulate important features of Hamilton’s study, but then synthesize it by rephrasing the study’s significance and relating it to their own work.

Keep your own voice

While the literature review presents others’ ideas, your voice (the writer’s) should remain front and center. Notice that Falk and Mills weave references to other sources into their own text, but they still maintain their own voice by starting and ending the paragraph with their own ideas and their own words. The sources support what Falk and Mills are saying.

Use caution when paraphrasing

When paraphrasing a source that is not your own, be sure to represent the author’s information or opinions accurately and in your own words. In the preceding example, Falk and Mills either directly refer in the text to the author of their source, such as Hamilton, or they provide ample notation in the text when the ideas they are mentioning are not their own, for example, Gastil’s. For more information, please see our handout on plagiarism .

Revise, revise, revise

Draft in hand? Now you’re ready to revise. Spending a lot of time revising is a wise idea, because your main objective is to present the material, not the argument. So check over your review again to make sure it follows the assignment and/or your outline. Then, just as you would for most other academic forms of writing, rewrite or rework the language of your review so that you’ve presented your information in the most concise manner possible. Be sure to use terminology familiar to your audience; get rid of unnecessary jargon or slang. Finally, double check that you’ve documented your sources and formatted the review appropriately for your discipline. For tips on the revising and editing process, see our handout on revising drafts .

Works consulted

We consulted these works while writing this handout. This is not a comprehensive list of resources on the handout’s topic, and we encourage you to do your own research to find additional publications. Please do not use this list as a model for the format of your own reference list, as it may not match the citation style you are using. For guidance on formatting citations, please see the UNC Libraries citation tutorial . We revise these tips periodically and welcome feedback.

Anson, Chris M., and Robert A. Schwegler. 2010. The Longman Handbook for Writers and Readers , 6th ed. New York: Longman.

Jones, Robert, Patrick Bizzaro, and Cynthia Selfe. 1997. The Harcourt Brace Guide to Writing in the Disciplines . New York: Harcourt Brace.

Lamb, Sandra E. 1998. How to Write It: A Complete Guide to Everything You’ll Ever Write . Berkeley: Ten Speed Press.

Rosen, Leonard J., and Laurence Behrens. 2003. The Allyn & Bacon Handbook , 5th ed. New York: Longman.

Troyka, Lynn Quittman, and Doug Hesse. 2016. Simon and Schuster Handbook for Writers , 11th ed. London: Pearson.

Make a Gift

The 5 C Guidelines

Choose a sign-in option, citation and embed code.

Producing good academic writing is a difficult skill to master, and writing for an academic audience is different than writing for other audiences.

As an academic writer, you must approach topics differently than you might as a journalist or creative author, and you must emphasize certain skills, such as writing clearly, and ignore other skills that you might have been taught in other contexts, such as using expressive imagery.

To introduce you to this world of academic writing, in this chapter I suggest that you should focus on five hierarchical characteristics of good writing, or the “5 Cs” of good academic writing, which include Clarity, Cogency, Conventionality, Completeness, and Concision. I will now explain each of these in more depth and then discuss tensions between them in writing for different academic audiences.

"Ambiguity is very interesting in writing; it's not very interesting in science." — Janna Levin

Many of us were taught in writing courses that ambiguity and obfuscation of meaning are laudable, because they make our writing seem more complex, deep, and witty. And many of our favorite novelists likely use ambiguity and other tricks to make their writing seem mysterious and complex.

In academic writing, though, these practices simply suggest that you don’t know what you’re talking about. For academics, writing is a way of uncovering truths and realities of the social and physical world, so we should say what we mean. We should make it clear, and say it so that it is impossible for our audience to misunderstand. If your imagined reader ever has to squint her eyes and muse “I wonder what the author really meant by this,” then you have failed. If your imagined reader ever smiles to herself and chuckles at your brilliant wordplay, then you have failed.

This is not to say that academic writing must be joyless and stodgy. It can be witty. It can be deep. But academic depth and wit come from the ideas portrayed through the words, not the words themselves. Too often, writers use ambiguity to hide sloppy thinking and beautiful language to hide destitute ideas. If you say something that could potentially be misunderstood, explain it. If a simpler word will do in place of a more complex one, then use the simple word. Don't be afraid of laying out complex ideas across multiple sentences or paragraphs, but use the space available to you to open up your mind to your reader — what exactly you are thinking, how you are thinking about it, and why.

If you use jargon, technical terms, or initialisms, then you should define or operationalize them. Defining a term means that you are relying upon someone else’s explanation of what the term means and are sticking with it (e.g., “Marwick defines ‘social media’ as…”). Operationalizing a term means that you are using a term that might mean multiple things but you are deciding to only use it in a very finite and specific manner (e.g., “In this paper, I use the word ‘engagement’ to mean…”).

In the case of initialisms, no reader should be expected to know what a PBL, SNS, LMS, CMS, or PBIS is by virtue of the letters themselves, and often even technical initialisms might have multiple meanings (e.g., PBL in education might refer to “project-based learning” or “problem-based learning”). So, when you use initialisms in your writing, define them at the outset (e.g., “positive behavioral interventions and supports (PBIS)”).

If you find yourself repeating the same words over and over again, that’s okay; don’t use variety in your language just for the sake of variety, because in highly technical academic fields every word carries with it technical baggage that you may not intend. If I’m talking about a “curriculum,” then I shouldn’t swap this out with “program,” “module,” “subject,” or “materials” just because I’m tired of saying “curriculum;” each of these words means something very different, and using these terms interchangeably just shows my reader that I don’t understand the differences between them.

In short, other types of writing often rely upon ambiguity and obfuscation to prevent their underlying thought processes from being examined, but examination of these processes is the whole purpose of academic writing. We are not poets, politicians, or preachers, whose primary goals are to be convincing or mystical; rather, we are fellow learners that can only learn together insofar as we can clearly reason together through a dialogic process of clear writing.

"Logic is the beginning of wisdom, not the end." — Leonard Nimoy

Your writing should follow a logical pattern or argument. Arguments generally follow a pattern of identifying one or more assumptions, providing evidence, and drawing a conclusion, such as:

• Assumption - Test creators should design tests to be equitable to all learners who may represent various demographic differences (e.g., race, ethnicity, gender);
• Evidence - Specific research studies show that some prominent tests exhibit inequitable outcomes for students based on race and ethnicity;
• Conclusion - Therefore, test creators should seek to understand why such tests are biased toward some students and should seek to remedy this issue.

Once a conclusion is drawn, it can then be used as an assumption for a subsequent argument. This means that the overarching argument for a paper actually consists of many micro-arguments made throughout the paper that build upon one another like bricks in a wall. Essentially, each paragraph in an academic paper should be treated as its own separate argument, and these arguments then build off of each other to construct the overarching argument.

To help reveal your logical structure, each paragraph should generally represent its own argument. It should start with a topic sentence, provide evidence, and then draw a conclusion that you then build upon in the next paragraph. This is what people often refer to as logical flow: moving from one idea to another without any unsubstantiated gaps.

Since you must substantiate each claim that you make in an academic paper, you should also be careful not to overstate claims (e.g., using superlative language) and not to make claims that you cannot provide evidence for. Oftentimes, this is done by toning down language (e.g., "project-based learning can be an effective pedagogical strategy" [toned down] vs. "project-based learning is the best pedagogical strategy" [superlative]) and using helping verbs, such as may, can, and might (e.g., "social media use may contribute to student depression" vs. "social media make kids depressed"). By toning down your language, you introduce possibilities for doubt (which academics should always be open to) and also prevent your argument from being invalidated by a single counter-instance (e.g., "my kid uses social media and is not depressed, therefore your argument is invalid").

Conventionality

"Writing [without structure] is like playing tennis with the net down." — Robert Frost

In order to understand each other and to know what to expect when we are reading a new manuscript, we need some conventions to provide uniformity. If every article you read had a unique structure, formatting, spacing, capitalization, font size, style, tone, and so forth, you would have increased difficulty comparing it to other work that had gone before.

However, if every time you approach a new piece of writing you know what to expect and where to find it, you will be able to more efficiently recognize where to go and what to look for in the paper. Rather than hunting for the research question, you will know that it will be found right before the methods section and that results will be provided right after that section. Similarly, when you see a bolded and centered line of text, you will know that this means that a new top-level (H1) section is beginning and are not left to wonder why the author made such a bold stylistic choice.

For these reasons, groups of academics have sets of guidelines that they agree to follow, and in our field we follow the American Psychological Association’s (APA) Handbook (version 6).  That handbook covers everything from structure and formatting to style and tone, and theses, dissertations, and class projects require students to follow these same guidelines to help ensure that student work is written in a way that will allow it to be published to a wider audience.

APA Template

To assist you in following APA guidelines, you can access this Google Document that has proper APA heading formats built into it. Then, click on the Styles dropdown > Options and save the APA styles as your default styles, thereby making any new documents that you create in Google Drive follow the APA conventions.

Completeness

"Our duty is to believe that for which we have sufficient evidence, and to suspend our judgment when we have not." — John Lubbock

Since the goal of academic work is to inform as broad an audience as possible and to stand up under the scrutiny of diverse sets of eyes, you must flesh out your writing to address all of the major questions and doubts that your readers might have. Thus, you must not only make an argument that might be convincing to a few people, but you must include sufficient detail and explanation to allow your writing to hold under the scrutiny of the most critical reader. That is, you must write for your greatest critic, not your greatest fan.

This is why academic publishing relies upon peer review. The assumption of peer review is that the best way to ensure quality of writing is to put an author’s work in the hands of dispassionate, unbiased, and diverse experts who typically do not know the identity of the author. This allows reviewers to be honest in their feedback and prevents them from relying upon their personal relationships with or knowledge of the author to fill any gaps (e.g., “I’ve heard of Dr. Avila, and she’s done great work before, so I’m assuming that she did good work here, too”). Rather, every piece of academic writing must stand or fall on its own merits and not on the author’s prior quality of work or reputation.

To do this, you must be explicit and detailed, assume that your reader does not know or trust you as the author, and treat every piece of academic writing as a completely self-contained, self-sustaining, self-validating artifact.

"Brevity is the soul of wit."— William Shakespeare

Most journals in education and other social sciences have strict article word limits of 4,000 to 7,000 words, or roughly 20 to 30 double-spaced pages. This historically has been done to accommodate publishing limitations, because printing pages in a journal is expensive, but even in a digital world, with practically zero publishing costs for adding additional words, imposing limits helps to reduce information overload on readers and to drive more readers to your work.

After all, a moderately-interested reader is much more likely to read a ten-page synopsis of your dissertation than the 200-page document itself, and only the most devoted of readers will stick with it after 20 pages or so. This means that you as the author must not only meet all of the requirements established by the previous guidelines, but you must do it on a strict word budget that avoids unnecessary detail or repetition.

Tensions and Relative Importance

As you’ve probably guessed by this point, the demands of each of these guidelines sometimes conflict with one another. For instance, being complete might mean that you are not very concise, and being clear might reveal the irrationality or poor cogency of your argument. At such times, you must prioritize the guidelines to determine which to emphasize and which to ignore.

Depending on the type of academic writing you are doing, the way that you prioritize guidelines may vary. For instance, a journal article will normally value concision over completeness, due to publishing word limits and trying to make the article as accessible as possible to a wide audience, while a systematic literature review for a thesis or dissertation will do the opposite, requiring students to reveal all of their understanding of their topic and how it is situated within the broader field so that their committees can be assured that they actually know what they are talking about.

Similarly, a quantitative empirical paper will rely heavily upon the conventions established by the paradigm (e.g., p-values, effect sizes), while a qualitative empirical paper will need to provide completeness in its descriptions to allow for trustworthiness and at times ignore conventions. Theoretical papers will also often necessarily defy some conventions in favor of laying out a clear and cogent argument, because the proposed ideas will be new and will not neatly fit within existing reporting approaches but will nonetheless need to be argued in a reasonable and compelling manner.

All this is to say that though these guidelines are all important, their relative importance to one another varies by the context of the writing as determined by purpose, audience, and methods.

Relative Importance of the 5 Cs in Different Settings

By focusing on these 5 Cs and each of their relative importance for your writing context, you can begin to approach your own work in a manner that is systematic and self-reflective. If writing a thesis, for instance, I should start by asking myself "Am I including all pertinent information and details? Is my meaning clear? Is my logic sound? Am I following APA guidelines? And am I careful not to repeat myself or belabor my point?" If writing a theoretical argument for a journal, on the other hand, I should ask myself "Am I within the word limit for the journal? Am I following the journal's stylistic guidelines? Is my argument strong? Is my meaning clear? And do I cover everything that I need to in order to preempt any concerns?"

When asking these questions, something always has to give, such as cutting that explanatory paragraph to get an article under the journal’s required word limit or adding an additional paragraph to make it clear to your thesis readers that you understand the implications of what you’re saying. Through it all, however, each of these guidelines is generally important to follow, and you can only justify ignoring one temporarily in those cases where tensions exist and your intended audience requires you to prioritize another guideline in its place.

Brigham Young University

This content is provided to you freely by EdTech Books.

Access it online or download it at https://edtechbooks.org/rapidwriting/5Cs .

• UConn Library
• Literature Review: The What, Why and How-to Guide
• Introduction

Literature Review: The What, Why and How-to Guide — Introduction

• Getting Started
• How to Pick a Topic
• Strategies to Find Sources
• Evaluating Sources & Lit. Reviews
• Tips for Writing Literature Reviews
• Writing Literature Review: Useful Sites
• Citation Resources
• Other Academic Writings

What are Literature Reviews?

So, what is a literature review? "A literature review is an account of what has been published on a topic by accredited scholars and researchers. In writing the literature review, your purpose is to convey to your reader what knowledge and ideas have been established on a topic, and what their strengths and weaknesses are. As a piece of writing, the literature review must be defined by a guiding concept (e.g., your research objective, the problem or issue you are discussing, or your argumentative thesis). It is not just a descriptive list of the material available, or a set of summaries." Taylor, D.  The literature review: A few tips on conducting it . University of Toronto Health Sciences Writing Centre.

Goals of Literature Reviews

What are the goals of creating a Literature Review?  A literature could be written to accomplish different aims:

• To develop a theory or evaluate an existing theory
• To summarize the historical or existing state of a research topic
• Identify a problem in a field of research 

Baumeister, R. F., & Leary, M. R. (1997). Writing narrative literature reviews .  Review of General Psychology , 1 (3), 311-320.

What kinds of sources require a Literature Review?

• A research paper assigned in a course
• A thesis or dissertation
• A grant proposal
• An article intended for publication in a journal

All these instances require you to collect what has been written about your research topic so that you can demonstrate how your own research sheds new light on the topic.

Types of Literature Reviews

What kinds of literature reviews are written?

Narrative review: The purpose of this type of review is to describe the current state of the research on a specific topic/research and to offer a critical analysis of the literature reviewed. Studies are grouped by research/theoretical categories, and themes and trends, strengths and weakness, and gaps are identified. The review ends with a conclusion section which summarizes the findings regarding the state of the research of the specific study, the gaps identify and if applicable, explains how the author's research will address gaps identify in the review and expand the knowledge on the topic reviewed.

• Example : Predictors and Outcomes of U.S. Quality Maternity Leave: A Review and Conceptual Framework:  10.1177/08948453211037398  

Systematic review : "The authors of a systematic review use a specific procedure to search the research literature, select the studies to include in their review, and critically evaluate the studies they find." (p. 139). Nelson, L. K. (2013). Research in Communication Sciences and Disorders . Plural Publishing.

• Example : The effect of leave policies on increasing fertility: a systematic review:  10.1057/s41599-022-01270-w

Meta-analysis : "Meta-analysis is a method of reviewing research findings in a quantitative fashion by transforming the data from individual studies into what is called an effect size and then pooling and analyzing this information. The basic goal in meta-analysis is to explain why different outcomes have occurred in different studies." (p. 197). Roberts, M. C., & Ilardi, S. S. (2003). Handbook of Research Methods in Clinical Psychology . Blackwell Publishing.

• Example : Employment Instability and Fertility in Europe: A Meta-Analysis:  10.1215/00703370-9164737

Meta-synthesis : "Qualitative meta-synthesis is a type of qualitative study that uses as data the findings from other qualitative studies linked by the same or related topic." (p.312). Zimmer, L. (2006). Qualitative meta-synthesis: A question of dialoguing with texts .  Journal of Advanced Nursing , 53 (3), 311-318.

• Example : Women’s perspectives on career successes and barriers: A qualitative meta-synthesis:  10.1177/05390184221113735

Literature Reviews in the Health Sciences

• UConn Health subject guide on systematic reviews Explanation of the different review types used in health sciences literature as well as tools to help you find the right review type
• << Previous: Getting Started
• Next: How to Pick a Topic >>
• Last Updated: Sep 21, 2022 2:16 PM
• URL: https://guides.lib.uconn.edu/literaturereview

Write a Literature Review

1. narrow your topic and select papers accordingly, 2. search for literature, 3. read the selected articles thoroughly and evaluate them, 4. organize the selected papers by looking for patterns and by developing subtopics, 5. develop a thesis or purpose statement, 6. write the paper, 7. review your work.

• Resources for Gathering and Reading the Literature
• Resources for Writing and Revising
• Other Useful Resources

Ask Us: Chat, email, visit or call

Get Assistance

The library offers a range of helpful services.  All of our appointments are free of charge and confidential.

• Book an appointment

Consider your specific area of study. Think about what interests you and what interests other researchers in your field.

Talk to your professor, brainstorm, and read lecture notes and recent issues of periodicals in the field.

Limit your scope to a smaller topic area (ie. focusing on France's role in WWII instead of focusing on WWII in general).

• Four Steps to Narrow Your Research Topic (Video) This 3-minute video provides instructions on how to narrow the focus of your research topic.
• Developing a Research Question + Worksheet Use this worksheet to develop, assess, and refine your research questions. There is also a downloadable PDF version.

Define your source selection criteria (ie. articles published between a specific date range, focusing on a specific geographic region, or using a specific methodology).

Using keywords, search a library database.

Reference lists of recent articles and reviews can lead to other useful papers.

Include any studies contrary to your point of view.

Evaluate and synthesize the studies' findings and conclusions.

Note the following:

• Assumptions some or most researchers seem to make
• Methodologies, testing procedures, subjects, material tested researchers use
• Experts in the field: names/labs that are frequently referenced
• Conflicting theories, results, methodologies
• Popularity of theories and how this has/has not changed over time
• Findings that are common/contested
• Important trends in the research
• The most influential theories

Tip: If your literature review is extensive, find a large table surface, and on it place post-it notes or filing cards to organize all your findings into categories.

• Move them around if you decide that (a) they fit better under different headings, or (b) you need to establish new topic headings.
• Develop headings/subheadings that reflect the major themes and patterns you detected

Write a one or two sentence statement summarizing the conclusion you have reached about the major trends and developments you see in the research that has been conducted on your subject.

• Templates for Writing Thesis Statements This template provides a two-step guide for writing thesis statements. There is also a downloadable PDF version.
• 5 Types of Thesis Statements Learn about five different types of thesis statements to help you choose the best type for your research. There is also a downloadable PDF version.
• 5 Questions to Strengthen Your Thesis Statement Follow these five steps to strengthen your thesis statements. There is also a downloadable PDF version.

Follow the organizational structure you developed above, including the headings and subheadings you constructed.

Make certain that each section links logically to the one before and after.

Structure your sections by themes or subtopics, not by individual theorists or researchers.

• Tip: If you find that each paragraph begins with a researcher's name, it might indicate that, instead of evaluating and comparing the research literature from an analytical point of view, you have simply described what research has been done.

Prioritize analysis over description.

• For example, look at the following two passages and note that Student A merely describes the literature, whereas Student B takes a more analytical and evaluative approach by comparing and contrasting. You can also see that this evaluative approach is well signaled by linguistic markers indicating logical connections (words such as "however," "moreover") and phrases such as "substantiates the claim that," which indicate supporting evidence and Student B's ability to synthesize knowledge.

Student A: Smith (2000) concludes that personal privacy in their living quarters is the most important factor in nursing home residents' perception of their autonomy. He suggests that the physical environment in the more public spaces of the building did not have much impact on their perceptions. Neither the layout of the building nor the activities available seem to make much difference. Jones and Johnstone make the claim that the need to control one's environment is a fundamental need of life (2001), and suggest that the approach of most institutions, which is to provide total care, may be as bad as no care at all. If people have no choices or think that they have none, they become depressed.

Student B: After studying residents and staff from two intermediate care facilities in Calgary, Alberta, Smith (2000) came to the conclusion that except for the amount of personal privacy available to residents, the physical environment of these institutions had minimal if any effect on their perceptions of control (autonomy). However, French (1998) and Haroon (2000) found that availability of private areas is not the only aspect of the physical environment that determines residents' autonomy. Haroon interviewed 115 residents from 32 different nursing homes known to have different levels of autonomy (2000). It was found that physical structures, such as standardized furniture, heating that could not be individually regulated, and no possession of a house key for residents limited their feelings of independence. Moreover, Hope (2002), who interviewed 225 residents from various nursing homes, substantiates the claim that characteristics of the institutional environment such as the extent of resources in the facility, as well as its location, are features which residents have indicated as being of great importance to their independence.

• How to Integrate Critical Voice into Your Literature Review (Video)
• Look at the topic sentences of each paragraph. If you were to read only these sentences, would you find that your paper presented a clear position, logically developed, from beginning to end? The topic sentences of each paragraph should indicate the main points of your literature review.
• Make an outline of each section of the paper and decide whether you need to add information, to delete irrelevant information, or to re-structure sections.
• Read your work out loud. That way you will be better able to identify where you need punctuation marks to signal pauses or divisions within sentences, where you have made grammatical errors, or where your sentences are unclear.
• Since the purpose of a literature review is to demonstrate that the writer is familiar with the important professional literature on the chosen subject, check to make certain that you have covered all of the important, up-to-date, and pertinent texts. In the sciences and some of the social sciences it is important that your literature be quite recent; this is not so important in the humanities.
• Make certain that all of the citations and references are correct and that you are referencing in the appropriate style for your discipline. If you are uncertain which style to use, ask your professor.
• Check to make sure that you have not plagiarized either by failing to cite a source of information, or by using words quoted directly from a source. (Usually if you take three or more words directly from another source, you should put those words within quotation marks, and cite the page.)
• Text should be written in a clear and concise academic style; it should not be descriptive in nature or use the language of everyday speech.
• There should be no grammatical or spelling errors.
• Sentences should flow smoothly and logically.
• << Previous: Start Here
• Next: Resources for Gathering and Reading the Literature >>
• Last Updated: Nov 8, 2022 9:19 AM
• URL: https://guides.lib.uoguelph.ca/LiteratureReview

Suggest an edit to this guide

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

Literature Review Research

Literature review, types of literature reviews.

• Finding information
• Additional Resources
• Explains the background of research on a topic
• Demonstrates why a topic is significant to a subject area
• Helps focus your own research questions or problems
• Discovers relationships between research studies/ideas
• Suggests unexplored ideas or populations
• Identifies major themes, concepts, and researchers on a topic
• Tests assumptions; may help counter preconceived ideas and remove unconscious bias.
• Identifies critical gaps, points of disagreement, or potentially flawed methodology or theoretical approaches

Argumentative Review This form examines literature selectively in order to support or refute an argument, deeply imbedded assumption, or philosophical problem already established in the literature. 

Integrative Review Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication.

Historical Review Historical reviews are focused on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review This approach provides a framework of understanding at different levels (i.e. those of theory, substantive fields, research approaches and data collection and analysis techniques), enables researchers to draw on a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork.

Systematic Review Uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyze data from the studies that are included in the review. Typically it focuses on a very specific empirical question.

Theoretical Review

Examines the theory that has accumulated in regard to an issue, concept, theory, phenomena. Helps to establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems.

* Kennedy, Mary M. "Defining a Literature."  Educational Researcher  36 (April 2007): 139-147.

All content in this section is from The Literature Review created by Dr. Robert Larabee USC

• Next: Finding information >>
• Last Updated: Oct 26, 2023 8:34 PM
• URL: https://guides.lib.odu.edu/literaturereview

An official website of the United States government

The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• Browse Titles

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.

Handbook of eHealth Evaluation: An Evidence-based Approach [Internet].

Chapter 9 methods for literature reviews.

Guy Paré and Spyros Kitsiou .

9.1. Introduction

Literature reviews play a critical role in scholarship because science remains, first and foremost, a cumulative endeavour ( vom Brocke et al., 2009 ). As in any academic discipline, rigorous knowledge syntheses are becoming indispensable in keeping up with an exponentially growing eHealth literature, assisting practitioners, academics, and graduate students in finding, evaluating, and synthesizing the contents of many empirical and conceptual papers. Among other methods, literature reviews are essential for: (a) identifying what has been written on a subject or topic; (b) determining the extent to which a specific research area reveals any interpretable trends or patterns; (c) aggregating empirical findings related to a narrow research question to support evidence-based practice; (d) generating new frameworks and theories; and (e) identifying topics or questions requiring more investigation ( Paré, Trudel, Jaana, & Kitsiou, 2015 ).

Literature reviews can take two major forms. The most prevalent one is the “literature review” or “background” section within a journal paper or a chapter in a graduate thesis. This section synthesizes the extant literature and usually identifies the gaps in knowledge that the empirical study addresses ( Sylvester, Tate, & Johnstone, 2013 ). It may also provide a theoretical foundation for the proposed study, substantiate the presence of the research problem, justify the research as one that contributes something new to the cumulated knowledge, or validate the methods and approaches for the proposed study ( Hart, 1998 ; Levy & Ellis, 2006 ).

The second form of literature review, which is the focus of this chapter, constitutes an original and valuable work of research in and of itself ( Paré et al., 2015 ). Rather than providing a base for a researcher’s own work, it creates a solid starting point for all members of the community interested in a particular area or topic ( Mulrow, 1987 ). The so-called “review article” is a journal-length paper which has an overarching purpose to synthesize the literature in a field, without collecting or analyzing any primary data ( Green, Johnson, & Adams, 2006 ).

When appropriately conducted, review articles represent powerful information sources for practitioners looking for state-of-the art evidence to guide their decision-making and work practices ( Paré et al., 2015 ). Further, high-quality reviews become frequently cited pieces of work which researchers seek out as a first clear outline of the literature when undertaking empirical studies ( Cooper, 1988 ; Rowe, 2014 ). Scholars who track and gauge the impact of articles have found that review papers are cited and downloaded more often than any other type of published article ( Cronin, Ryan, & Coughlan, 2008 ; Montori, Wilczynski, Morgan, Haynes, & Hedges, 2003 ; Patsopoulos, Analatos, & Ioannidis, 2005 ). The reason for their popularity may be the fact that reading the review enables one to have an overview, if not a detailed knowledge of the area in question, as well as references to the most useful primary sources ( Cronin et al., 2008 ). Although they are not easy to conduct, the commitment to complete a review article provides a tremendous service to one’s academic community ( Paré et al., 2015 ; Petticrew & Roberts, 2006 ). Most, if not all, peer-reviewed journals in the fields of medical informatics publish review articles of some type.

The main objectives of this chapter are fourfold: (a) to provide an overview of the major steps and activities involved in conducting a stand-alone literature review; (b) to describe and contrast the different types of review articles that can contribute to the eHealth knowledge base; (c) to illustrate each review type with one or two examples from the eHealth literature; and (d) to provide a series of recommendations for prospective authors of review articles in this domain.

9.2. Overview of the Literature Review Process and Steps

As explained in Templier and Paré (2015) , there are six generic steps involved in conducting a review article:

• formulating the research question(s) and objective(s),
• searching the extant literature,
• screening for inclusion,
• assessing the quality of primary studies,
• extracting data, and
• analyzing data.

Although these steps are presented here in sequential order, one must keep in mind that the review process can be iterative and that many activities can be initiated during the planning stage and later refined during subsequent phases ( Finfgeld-Connett & Johnson, 2013 ; Kitchenham & Charters, 2007 ).

Formulating the research question(s) and objective(s): As a first step, members of the review team must appropriately justify the need for the review itself ( Petticrew & Roberts, 2006 ), identify the review’s main objective(s) ( Okoli & Schabram, 2010 ), and define the concepts or variables at the heart of their synthesis ( Cooper & Hedges, 2009 ; Webster & Watson, 2002 ). Importantly, they also need to articulate the research question(s) they propose to investigate ( Kitchenham & Charters, 2007 ). In this regard, we concur with Jesson, Matheson, and Lacey (2011) that clearly articulated research questions are key ingredients that guide the entire review methodology; they underscore the type of information that is needed, inform the search for and selection of relevant literature, and guide or orient the subsequent analysis. Searching the extant literature: The next step consists of searching the literature and making decisions about the suitability of material to be considered in the review ( Cooper, 1988 ). There exist three main coverage strategies. First, exhaustive coverage means an effort is made to be as comprehensive as possible in order to ensure that all relevant studies, published and unpublished, are included in the review and, thus, conclusions are based on this all-inclusive knowledge base. The second type of coverage consists of presenting materials that are representative of most other works in a given field or area. Often authors who adopt this strategy will search for relevant articles in a small number of top-tier journals in a field ( Paré et al., 2015 ). In the third strategy, the review team concentrates on prior works that have been central or pivotal to a particular topic. This may include empirical studies or conceptual papers that initiated a line of investigation, changed how problems or questions were framed, introduced new methods or concepts, or engendered important debate ( Cooper, 1988 ). Screening for inclusion: The following step consists of evaluating the applicability of the material identified in the preceding step ( Levy & Ellis, 2006 ; vom Brocke et al., 2009 ). Once a group of potential studies has been identified, members of the review team must screen them to determine their relevance ( Petticrew & Roberts, 2006 ). A set of predetermined rules provides a basis for including or excluding certain studies. This exercise requires a significant investment on the part of researchers, who must ensure enhanced objectivity and avoid biases or mistakes. As discussed later in this chapter, for certain types of reviews there must be at least two independent reviewers involved in the screening process and a procedure to resolve disagreements must also be in place ( Liberati et al., 2009 ; Shea et al., 2009 ). Assessing the quality of primary studies: In addition to screening material for inclusion, members of the review team may need to assess the scientific quality of the selected studies, that is, appraise the rigour of the research design and methods. Such formal assessment, which is usually conducted independently by at least two coders, helps members of the review team refine which studies to include in the final sample, determine whether or not the differences in quality may affect their conclusions, or guide how they analyze the data and interpret the findings ( Petticrew & Roberts, 2006 ). Ascribing quality scores to each primary study or considering through domain-based evaluations which study components have or have not been designed and executed appropriately makes it possible to reflect on the extent to which the selected study addresses possible biases and maximizes validity ( Shea et al., 2009 ). Extracting data: The following step involves gathering or extracting applicable information from each primary study included in the sample and deciding what is relevant to the problem of interest ( Cooper & Hedges, 2009 ). Indeed, the type of data that should be recorded mainly depends on the initial research questions ( Okoli & Schabram, 2010 ). However, important information may also be gathered about how, when, where and by whom the primary study was conducted, the research design and methods, or qualitative/quantitative results ( Cooper & Hedges, 2009 ). Analyzing and synthesizing data : As a final step, members of the review team must collate, summarize, aggregate, organize, and compare the evidence extracted from the included studies. The extracted data must be presented in a meaningful way that suggests a new contribution to the extant literature ( Jesson et al., 2011 ). Webster and Watson (2002) warn researchers that literature reviews should be much more than lists of papers and should provide a coherent lens to make sense of extant knowledge on a given topic. There exist several methods and techniques for synthesizing quantitative (e.g., frequency analysis, meta-analysis) and qualitative (e.g., grounded theory, narrative analysis, meta-ethnography) evidence ( Dixon-Woods, Agarwal, Jones, Young, & Sutton, 2005 ; Thomas & Harden, 2008 ).

9.3. Types of Review Articles and Brief Illustrations

EHealth researchers have at their disposal a number of approaches and methods for making sense out of existing literature, all with the purpose of casting current research findings into historical contexts or explaining contradictions that might exist among a set of primary research studies conducted on a particular topic. Our classification scheme is largely inspired from Paré and colleagues’ (2015) typology. Below we present and illustrate those review types that we feel are central to the growth and development of the eHealth domain.

9.3.1. Narrative Reviews

The narrative review is the “traditional” way of reviewing the extant literature and is skewed towards a qualitative interpretation of prior knowledge ( Sylvester et al., 2013 ). Put simply, a narrative review attempts to summarize or synthesize what has been written on a particular topic but does not seek generalization or cumulative knowledge from what is reviewed ( Davies, 2000 ; Green et al., 2006 ). Instead, the review team often undertakes the task of accumulating and synthesizing the literature to demonstrate the value of a particular point of view ( Baumeister & Leary, 1997 ). As such, reviewers may selectively ignore or limit the attention paid to certain studies in order to make a point. In this rather unsystematic approach, the selection of information from primary articles is subjective, lacks explicit criteria for inclusion and can lead to biased interpretations or inferences ( Green et al., 2006 ). There are several narrative reviews in the particular eHealth domain, as in all fields, which follow such an unstructured approach ( Silva et al., 2015 ; Paul et al., 2015 ).

Despite these criticisms, this type of review can be very useful in gathering together a volume of literature in a specific subject area and synthesizing it. As mentioned above, its primary purpose is to provide the reader with a comprehensive background for understanding current knowledge and highlighting the significance of new research ( Cronin et al., 2008 ). Faculty like to use narrative reviews in the classroom because they are often more up to date than textbooks, provide a single source for students to reference, and expose students to peer-reviewed literature ( Green et al., 2006 ). For researchers, narrative reviews can inspire research ideas by identifying gaps or inconsistencies in a body of knowledge, thus helping researchers to determine research questions or formulate hypotheses. Importantly, narrative reviews can also be used as educational articles to bring practitioners up to date with certain topics of issues ( Green et al., 2006 ).

Recently, there have been several efforts to introduce more rigour in narrative reviews that will elucidate common pitfalls and bring changes into their publication standards. Information systems researchers, among others, have contributed to advancing knowledge on how to structure a “traditional” review. For instance, Levy and Ellis (2006) proposed a generic framework for conducting such reviews. Their model follows the systematic data processing approach comprised of three steps, namely: (a) literature search and screening; (b) data extraction and analysis; and (c) writing the literature review. They provide detailed and very helpful instructions on how to conduct each step of the review process. As another methodological contribution, vom Brocke et al. (2009) offered a series of guidelines for conducting literature reviews, with a particular focus on how to search and extract the relevant body of knowledge. Last, Bandara, Miskon, and Fielt (2011) proposed a structured, predefined and tool-supported method to identify primary studies within a feasible scope, extract relevant content from identified articles, synthesize and analyze the findings, and effectively write and present the results of the literature review. We highly recommend that prospective authors of narrative reviews consult these useful sources before embarking on their work.

Darlow and Wen (2015) provide a good example of a highly structured narrative review in the eHealth field. These authors synthesized published articles that describe the development process of mobile health ( m-health ) interventions for patients’ cancer care self-management. As in most narrative reviews, the scope of the research questions being investigated is broad: (a) how development of these systems are carried out; (b) which methods are used to investigate these systems; and (c) what conclusions can be drawn as a result of the development of these systems. To provide clear answers to these questions, a literature search was conducted on six electronic databases and Google Scholar . The search was performed using several terms and free text words, combining them in an appropriate manner. Four inclusion and three exclusion criteria were utilized during the screening process. Both authors independently reviewed each of the identified articles to determine eligibility and extract study information. A flow diagram shows the number of studies identified, screened, and included or excluded at each stage of study selection. In terms of contributions, this review provides a series of practical recommendations for m-health intervention development.

9.3.2. Descriptive or Mapping Reviews

The primary goal of a descriptive review is to determine the extent to which a body of knowledge in a particular research topic reveals any interpretable pattern or trend with respect to pre-existing propositions, theories, methodologies or findings ( King & He, 2005 ; Paré et al., 2015 ). In contrast with narrative reviews, descriptive reviews follow a systematic and transparent procedure, including searching, screening and classifying studies ( Petersen, Vakkalanka, & Kuzniarz, 2015 ). Indeed, structured search methods are used to form a representative sample of a larger group of published works ( Paré et al., 2015 ). Further, authors of descriptive reviews extract from each study certain characteristics of interest, such as publication year, research methods, data collection techniques, and direction or strength of research outcomes (e.g., positive, negative, or non-significant) in the form of frequency analysis to produce quantitative results ( Sylvester et al., 2013 ). In essence, each study included in a descriptive review is treated as the unit of analysis and the published literature as a whole provides a database from which the authors attempt to identify any interpretable trends or draw overall conclusions about the merits of existing conceptualizations, propositions, methods or findings ( Paré et al., 2015 ). In doing so, a descriptive review may claim that its findings represent the state of the art in a particular domain ( King & He, 2005 ).

In the fields of health sciences and medical informatics, reviews that focus on examining the range, nature and evolution of a topic area are described by Anderson, Allen, Peckham, and Goodwin (2008) as mapping reviews . Like descriptive reviews, the research questions are generic and usually relate to publication patterns and trends. There is no preconceived plan to systematically review all of the literature although this can be done. Instead, researchers often present studies that are representative of most works published in a particular area and they consider a specific time frame to be mapped.

An example of this approach in the eHealth domain is offered by DeShazo, Lavallie, and Wolf (2009). The purpose of this descriptive or mapping review was to characterize publication trends in the medical informatics literature over a 20-year period (1987 to 2006). To achieve this ambitious objective, the authors performed a bibliometric analysis of medical informatics citations indexed in medline using publication trends, journal frequencies, impact factors, Medical Subject Headings (MeSH) term frequencies, and characteristics of citations. Findings revealed that there were over 77,000 medical informatics articles published during the covered period in numerous journals and that the average annual growth rate was 12%. The MeSH term analysis also suggested a strong interdisciplinary trend. Finally, average impact scores increased over time with two notable growth periods. Overall, patterns in research outputs that seem to characterize the historic trends and current components of the field of medical informatics suggest it may be a maturing discipline (DeShazo et al., 2009).

9.3.3. Scoping Reviews

Scoping reviews attempt to provide an initial indication of the potential size and nature of the extant literature on an emergent topic (Arksey & O’Malley, 2005; Daudt, van Mossel, & Scott, 2013 ; Levac, Colquhoun, & O’Brien, 2010). A scoping review may be conducted to examine the extent, range and nature of research activities in a particular area, determine the value of undertaking a full systematic review (discussed next), or identify research gaps in the extant literature ( Paré et al., 2015 ). In line with their main objective, scoping reviews usually conclude with the presentation of a detailed research agenda for future works along with potential implications for both practice and research.

Unlike narrative and descriptive reviews, the whole point of scoping the field is to be as comprehensive as possible, including grey literature (Arksey & O’Malley, 2005). Inclusion and exclusion criteria must be established to help researchers eliminate studies that are not aligned with the research questions. It is also recommended that at least two independent coders review abstracts yielded from the search strategy and then the full articles for study selection ( Daudt et al., 2013 ). The synthesized evidence from content or thematic analysis is relatively easy to present in tabular form (Arksey & O’Malley, 2005; Thomas & Harden, 2008 ).

One of the most highly cited scoping reviews in the eHealth domain was published by Archer, Fevrier-Thomas, Lokker, McKibbon, and Straus (2011) . These authors reviewed the existing literature on personal health record ( phr ) systems including design, functionality, implementation, applications, outcomes, and benefits. Seven databases were searched from 1985 to March 2010. Several search terms relating to phr s were used during this process. Two authors independently screened titles and abstracts to determine inclusion status. A second screen of full-text articles, again by two independent members of the research team, ensured that the studies described phr s. All in all, 130 articles met the criteria and their data were extracted manually into a database. The authors concluded that although there is a large amount of survey, observational, cohort/panel, and anecdotal evidence of phr benefits and satisfaction for patients, more research is needed to evaluate the results of phr implementations. Their in-depth analysis of the literature signalled that there is little solid evidence from randomized controlled trials or other studies through the use of phr s. Hence, they suggested that more research is needed that addresses the current lack of understanding of optimal functionality and usability of these systems, and how they can play a beneficial role in supporting patient self-management ( Archer et al., 2011 ).

9.3.4. Forms of Aggregative Reviews

Healthcare providers, practitioners, and policy-makers are nowadays overwhelmed with large volumes of information, including research-based evidence from numerous clinical trials and evaluation studies, assessing the effectiveness of health information technologies and interventions ( Ammenwerth & de Keizer, 2004 ; Deshazo et al., 2009 ). It is unrealistic to expect that all these disparate actors will have the time, skills, and necessary resources to identify the available evidence in the area of their expertise and consider it when making decisions. Systematic reviews that involve the rigorous application of scientific strategies aimed at limiting subjectivity and bias (i.e., systematic and random errors) can respond to this challenge.

Systematic reviews attempt to aggregate, appraise, and synthesize in a single source all empirical evidence that meet a set of previously specified eligibility criteria in order to answer a clearly formulated and often narrow research question on a particular topic of interest to support evidence-based practice ( Liberati et al., 2009 ). They adhere closely to explicit scientific principles ( Liberati et al., 2009 ) and rigorous methodological guidelines (Higgins & Green, 2008) aimed at reducing random and systematic errors that can lead to deviations from the truth in results or inferences. The use of explicit methods allows systematic reviews to aggregate a large body of research evidence, assess whether effects or relationships are in the same direction and of the same general magnitude, explain possible inconsistencies between study results, and determine the strength of the overall evidence for every outcome of interest based on the quality of included studies and the general consistency among them ( Cook, Mulrow, & Haynes, 1997 ). The main procedures of a systematic review involve:

• Formulating a review question and developing a search strategy based on explicit inclusion criteria for the identification of eligible studies (usually described in the context of a detailed review protocol).
• Searching for eligible studies using multiple databases and information sources, including grey literature sources, without any language restrictions.
• Selecting studies, extracting data, and assessing risk of bias in a duplicate manner using two independent reviewers to avoid random or systematic errors in the process.
• Analyzing data using quantitative or qualitative methods.
• Presenting results in summary of findings tables.
• Interpreting results and drawing conclusions.

Many systematic reviews, but not all, use statistical methods to combine the results of independent studies into a single quantitative estimate or summary effect size. Known as meta-analyses , these reviews use specific data extraction and statistical techniques (e.g., network, frequentist, or Bayesian meta-analyses) to calculate from each study by outcome of interest an effect size along with a confidence interval that reflects the degree of uncertainty behind the point estimate of effect ( Borenstein, Hedges, Higgins, & Rothstein, 2009 ; Deeks, Higgins, & Altman, 2008 ). Subsequently, they use fixed or random-effects analysis models to combine the results of the included studies, assess statistical heterogeneity, and calculate a weighted average of the effect estimates from the different studies, taking into account their sample sizes. The summary effect size is a value that reflects the average magnitude of the intervention effect for a particular outcome of interest or, more generally, the strength of a relationship between two variables across all studies included in the systematic review. By statistically combining data from multiple studies, meta-analyses can create more precise and reliable estimates of intervention effects than those derived from individual studies alone, when these are examined independently as discrete sources of information.

The review by Gurol-Urganci, de Jongh, Vodopivec-Jamsek, Atun, and Car (2013) on the effects of mobile phone messaging reminders for attendance at healthcare appointments is an illustrative example of a high-quality systematic review with meta-analysis. Missed appointments are a major cause of inefficiency in healthcare delivery with substantial monetary costs to health systems. These authors sought to assess whether mobile phone-based appointment reminders delivered through Short Message Service ( sms ) or Multimedia Messaging Service ( mms ) are effective in improving rates of patient attendance and reducing overall costs. To this end, they conducted a comprehensive search on multiple databases using highly sensitive search strategies without language or publication-type restrictions to identify all rct s that are eligible for inclusion. In order to minimize the risk of omitting eligible studies not captured by the original search, they supplemented all electronic searches with manual screening of trial registers and references contained in the included studies. Study selection, data extraction, and risk of bias assessments were performed inde­­pen­dently by two coders using standardized methods to ensure consistency and to eliminate potential errors. Findings from eight rct s involving 6,615 participants were pooled into meta-analyses to calculate the magnitude of effects that mobile text message reminders have on the rate of attendance at healthcare appointments compared to no reminders and phone call reminders.

Meta-analyses are regarded as powerful tools for deriving meaningful conclusions. However, there are situations in which it is neither reasonable nor appropriate to pool studies together using meta-analytic methods simply because there is extensive clinical heterogeneity between the included studies or variation in measurement tools, comparisons, or outcomes of interest. In these cases, systematic reviews can use qualitative synthesis methods such as vote counting, content analysis, classification schemes and tabulations, as an alternative approach to narratively synthesize the results of the independent studies included in the review. This form of review is known as qualitative systematic review.

A rigorous example of one such review in the eHealth domain is presented by Mickan, Atherton, Roberts, Heneghan, and Tilson (2014) on the use of handheld computers by healthcare professionals and their impact on access to information and clinical decision-making. In line with the methodological guide­lines for systematic reviews, these authors: (a) developed and registered with prospero ( www.crd.york.ac.uk/ prospero / ) an a priori review protocol; (b) conducted comprehensive searches for eligible studies using multiple databases and other supplementary strategies (e.g., forward searches); and (c) subsequently carried out study selection, data extraction, and risk of bias assessments in a duplicate manner to eliminate potential errors in the review process. Heterogeneity between the included studies in terms of reported outcomes and measures precluded the use of meta-analytic methods. To this end, the authors resorted to using narrative analysis and synthesis to describe the effectiveness of handheld computers on accessing information for clinical knowledge, adherence to safety and clinical quality guidelines, and diagnostic decision-making.

In recent years, the number of systematic reviews in the field of health informatics has increased considerably. Systematic reviews with discordant findings can cause great confusion and make it difficult for decision-makers to interpret the review-level evidence ( Moher, 2013 ). Therefore, there is a growing need for appraisal and synthesis of prior systematic reviews to ensure that decision-making is constantly informed by the best available accumulated evidence. Umbrella reviews , also known as overviews of systematic reviews, are tertiary types of evidence synthesis that aim to accomplish this; that is, they aim to compare and contrast findings from multiple systematic reviews and meta-analyses ( Becker & Oxman, 2008 ). Umbrella reviews generally adhere to the same principles and rigorous methodological guidelines used in systematic reviews. However, the unit of analysis in umbrella reviews is the systematic review rather than the primary study ( Becker & Oxman, 2008 ). Unlike systematic reviews that have a narrow focus of inquiry, umbrella reviews focus on broader research topics for which there are several potential interventions ( Smith, Devane, Begley, & Clarke, 2011 ). A recent umbrella review on the effects of home telemonitoring interventions for patients with heart failure critically appraised, compared, and synthesized evidence from 15 systematic reviews to investigate which types of home telemonitoring technologies and forms of interventions are more effective in reducing mortality and hospital admissions ( Kitsiou, Paré, & Jaana, 2015 ).

9.3.5. Realist Reviews

Realist reviews are theory-driven interpretative reviews developed to inform, enhance, or supplement conventional systematic reviews by making sense of heterogeneous evidence about complex interventions applied in diverse contexts in a way that informs policy decision-making ( Greenhalgh, Wong, Westhorp, & Pawson, 2011 ). They originated from criticisms of positivist systematic reviews which centre on their “simplistic” underlying assumptions ( Oates, 2011 ). As explained above, systematic reviews seek to identify causation. Such logic is appropriate for fields like medicine and education where findings of randomized controlled trials can be aggregated to see whether a new treatment or intervention does improve outcomes. However, many argue that it is not possible to establish such direct causal links between interventions and outcomes in fields such as social policy, management, and information systems where for any intervention there is unlikely to be a regular or consistent outcome ( Oates, 2011 ; Pawson, 2006 ; Rousseau, Manning, & Denyer, 2008 ).

To circumvent these limitations, Pawson, Greenhalgh, Harvey, and Walshe (2005) have proposed a new approach for synthesizing knowledge that seeks to unpack the mechanism of how “complex interventions” work in particular contexts. The basic research question — what works? — which is usually associated with systematic reviews changes to: what is it about this intervention that works, for whom, in what circumstances, in what respects and why? Realist reviews have no particular preference for either quantitative or qualitative evidence. As a theory-building approach, a realist review usually starts by articulating likely underlying mechanisms and then scrutinizes available evidence to find out whether and where these mechanisms are applicable ( Shepperd et al., 2009 ). Primary studies found in the extant literature are viewed as case studies which can test and modify the initial theories ( Rousseau et al., 2008 ).

The main objective pursued in the realist review conducted by Otte-Trojel, de Bont, Rundall, and van de Klundert (2014) was to examine how patient portals contribute to health service delivery and patient outcomes. The specific goals were to investigate how outcomes are produced and, most importantly, how variations in outcomes can be explained. The research team started with an exploratory review of background documents and research studies to identify ways in which patient portals may contribute to health service delivery and patient outcomes. The authors identified six main ways which represent “educated guesses” to be tested against the data in the evaluation studies. These studies were identified through a formal and systematic search in four databases between 2003 and 2013. Two members of the research team selected the articles using a pre-established list of inclusion and exclusion criteria and following a two-step procedure. The authors then extracted data from the selected articles and created several tables, one for each outcome category. They organized information to bring forward those mechanisms where patient portals contribute to outcomes and the variation in outcomes across different contexts.

9.3.6. Critical Reviews

Lastly, critical reviews aim to provide a critical evaluation and interpretive analysis of existing literature on a particular topic of interest to reveal strengths, weaknesses, contradictions, controversies, inconsistencies, and/or other important issues with respect to theories, hypotheses, research methods or results ( Baumeister & Leary, 1997 ; Kirkevold, 1997 ). Unlike other review types, critical reviews attempt to take a reflective account of the research that has been done in a particular area of interest, and assess its credibility by using appraisal instruments or critical interpretive methods. In this way, critical reviews attempt to constructively inform other scholars about the weaknesses of prior research and strengthen knowledge development by giving focus and direction to studies for further improvement ( Kirkevold, 1997 ).

Kitsiou, Paré, and Jaana (2013) provide an example of a critical review that assessed the methodological quality of prior systematic reviews of home telemonitoring studies for chronic patients. The authors conducted a comprehensive search on multiple databases to identify eligible reviews and subsequently used a validated instrument to conduct an in-depth quality appraisal. Results indicate that the majority of systematic reviews in this particular area suffer from important methodological flaws and biases that impair their internal validity and limit their usefulness for clinical and decision-making purposes. To this end, they provide a number of recommendations to strengthen knowledge development towards improving the design and execution of future reviews on home telemonitoring.

9.4. Summary

Table 9.1 outlines the main types of literature reviews that were described in the previous sub-sections and summarizes the main characteristics that distinguish one review type from another. It also includes key references to methodological guidelines and useful sources that can be used by eHealth scholars and researchers for planning and developing reviews.

Typology of Literature Reviews (adapted from Paré et al., 2015).

As shown in Table 9.1 , each review type addresses different kinds of research questions or objectives, which subsequently define and dictate the methods and approaches that need to be used to achieve the overarching goal(s) of the review. For example, in the case of narrative reviews, there is greater flexibility in searching and synthesizing articles ( Green et al., 2006 ). Researchers are often relatively free to use a diversity of approaches to search, identify, and select relevant scientific articles, describe their operational characteristics, present how the individual studies fit together, and formulate conclusions. On the other hand, systematic reviews are characterized by their high level of systematicity, rigour, and use of explicit methods, based on an “a priori” review plan that aims to minimize bias in the analysis and synthesis process (Higgins & Green, 2008). Some reviews are exploratory in nature (e.g., scoping/mapping reviews), whereas others may be conducted to discover patterns (e.g., descriptive reviews) or involve a synthesis approach that may include the critical analysis of prior research ( Paré et al., 2015 ). Hence, in order to select the most appropriate type of review, it is critical to know before embarking on a review project, why the research synthesis is conducted and what type of methods are best aligned with the pursued goals.

9.5. Concluding Remarks

In light of the increased use of evidence-based practice and research generating stronger evidence ( Grady et al., 2011 ; Lyden et al., 2013 ), review articles have become essential tools for summarizing, synthesizing, integrating or critically appraising prior knowledge in the eHealth field. As mentioned earlier, when rigorously conducted review articles represent powerful information sources for eHealth scholars and practitioners looking for state-of-the-art evidence. The typology of literature reviews we used herein will allow eHealth researchers, graduate students and practitioners to gain a better understanding of the similarities and differences between review types.

We must stress that this classification scheme does not privilege any specific type of review as being of higher quality than another ( Paré et al., 2015 ). As explained above, each type of review has its own strengths and limitations. Having said that, we realize that the methodological rigour of any review — be it qualitative, quantitative or mixed — is a critical aspect that should be considered seriously by prospective authors. In the present context, the notion of rigour refers to the reliability and validity of the review process described in section 9.2. For one thing, reliability is related to the reproducibility of the review process and steps, which is facilitated by a comprehensive documentation of the literature search process, extraction, coding and analysis performed in the review. Whether the search is comprehensive or not, whether it involves a methodical approach for data extraction and synthesis or not, it is important that the review documents in an explicit and transparent manner the steps and approach that were used in the process of its development. Next, validity characterizes the degree to which the review process was conducted appropriately. It goes beyond documentation and reflects decisions related to the selection of the sources, the search terms used, the period of time covered, the articles selected in the search, and the application of backward and forward searches ( vom Brocke et al., 2009 ). In short, the rigour of any review article is reflected by the explicitness of its methods (i.e., transparency) and the soundness of the approach used. We refer those interested in the concepts of rigour and quality to the work of Templier and Paré (2015) which offers a detailed set of methodological guidelines for conducting and evaluating various types of review articles.

To conclude, our main objective in this chapter was to demystify the various types of literature reviews that are central to the continuous development of the eHealth field. It is our hope that our descriptive account will serve as a valuable source for those conducting, evaluating or using reviews in this important and growing domain.

• Ammenwerth E., de Keizer N. An inventory of evaluation studies of information technology in health care. Trends in evaluation research, 1982-2002. International Journal of Medical Informatics. 2004; 44 (1):44–56. [ PubMed : 15778794 ]
• Anderson S., Allen P., Peckham S., Goodwin N. Asking the right questions: scoping studies in the commissioning of research on the organisation and delivery of health services. Health Research Policy and Systems. 2008; 6 (7):1–12. [ PMC free article : PMC2500008 ] [ PubMed : 18613961 ] [ CrossRef ]
• Archer N., Fevrier-Thomas U., Lokker C., McKibbon K. A., Straus S.E. Personal health records: a scoping review. Journal of American Medical Informatics Association. 2011; 18 (4):515–522. [ PMC free article : PMC3128401 ] [ PubMed : 21672914 ]
• Arksey H., O’Malley L. Scoping studies: towards a methodological framework. International Journal of Social Research Methodology. 2005; 8 (1):19–32.
• A systematic, tool-supported method for conducting literature reviews in information systems. Paper presented at the Proceedings of the 19th European Conference on Information Systems ( ecis 2011); June 9 to 11; Helsinki, Finland. 2011.
• Baumeister R. F., Leary M.R. Writing narrative literature reviews. Review of General Psychology. 1997; 1 (3):311–320.
• Becker L. A., Oxman A.D. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Overviews of reviews; pp. 607–631.
• Borenstein M., Hedges L., Higgins J., Rothstein H. Introduction to meta-analysis. Hoboken, nj : John Wiley & Sons Inc; 2009.
• Cook D. J., Mulrow C. D., Haynes B. Systematic reviews: Synthesis of best evidence for clinical decisions. Annals of Internal Medicine. 1997; 126 (5):376–380. [ PubMed : 9054282 ]
• Cooper H., Hedges L.V. In: The handbook of research synthesis and meta-analysis. 2nd ed. Cooper H., Hedges L. V., Valentine J. C., editors. New York: Russell Sage Foundation; 2009. Research synthesis as a scientific process; pp. 3–17.
• Cooper H. M. Organizing knowledge syntheses: A taxonomy of literature reviews. Knowledge in Society. 1988; 1 (1):104–126.
• Cronin P., Ryan F., Coughlan M. Undertaking a literature review: a step-by-step approach. British Journal of Nursing. 2008; 17 (1):38–43. [ PubMed : 18399395 ]
• Darlow S., Wen K.Y. Development testing of mobile health interventions for cancer patient self-management: A review. Health Informatics Journal. 2015 (online before print). [ PubMed : 25916831 ] [ CrossRef ]
• Daudt H. M., van Mossel C., Scott S.J. Enhancing the scoping study methodology: a large, inter-professional team’s experience with Arksey and O’Malley’s framework. bmc Medical Research Methodology. 2013; 13 :48. [ PMC free article : PMC3614526 ] [ PubMed : 23522333 ] [ CrossRef ]
• Davies P. The relevance of systematic reviews to educational policy and practice. Oxford Review of Education. 2000; 26 (3-4):365–378.
• Deeks J. J., Higgins J. P. T., Altman D.G. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Analysing data and undertaking meta-analyses; pp. 243–296.
• Deshazo J. P., Lavallie D. L., Wolf F.M. Publication trends in the medical informatics literature: 20 years of “Medical Informatics” in mesh . bmc Medical Informatics and Decision Making. 2009; 9 :7. [ PMC free article : PMC2652453 ] [ PubMed : 19159472 ] [ CrossRef ]
• Dixon-Woods M., Agarwal S., Jones D., Young B., Sutton A. Synthesising qualitative and quantitative evidence: a review of possible methods. Journal of Health Services Research and Policy. 2005; 10 (1):45–53. [ PubMed : 15667704 ]
• Finfgeld-Connett D., Johnson E.D. Literature search strategies for conducting knowledge-building and theory-generating qualitative systematic reviews. Journal of Advanced Nursing. 2013; 69 (1):194–204. [ PMC free article : PMC3424349 ] [ PubMed : 22591030 ]
• Grady B., Myers K. M., Nelson E. L., Belz N., Bennett L., Carnahan L. … Guidelines Working Group. Evidence-based practice for telemental health. Telemedicine Journal and E Health. 2011; 17 (2):131–148. [ PubMed : 21385026 ]
• Green B. N., Johnson C. D., Adams A. Writing narrative literature reviews for peer-reviewed journals: secrets of the trade. Journal of Chiropractic Medicine. 2006; 5 (3):101–117. [ PMC free article : PMC2647067 ] [ PubMed : 19674681 ]
• Greenhalgh T., Wong G., Westhorp G., Pawson R. Protocol–realist and meta-narrative evidence synthesis: evolving standards ( rameses ). bmc Medical Research Methodology. 2011; 11 :115. [ PMC free article : PMC3173389 ] [ PubMed : 21843376 ]
• Gurol-Urganci I., de Jongh T., Vodopivec-Jamsek V., Atun R., Car J. Mobile phone messaging reminders for attendance at healthcare appointments. Cochrane Database System Review. 2013; 12 cd 007458. [ PMC free article : PMC6485985 ] [ PubMed : 24310741 ] [ CrossRef ]
• Hart C. Doing a literature review: Releasing the social science research imagination. London: SAGE Publications; 1998.
• Higgins J. P. T., Green S., editors. Cochrane handbook for systematic reviews of interventions: Cochrane book series. Hoboken, nj : Wiley-Blackwell; 2008.
• Jesson J., Matheson L., Lacey F.M. Doing your literature review: traditional and systematic techniques. Los Angeles & London: SAGE Publications; 2011.
• King W. R., He J. Understanding the role and methods of meta-analysis in IS research. Communications of the Association for Information Systems. 2005; 16 :1.
• Kirkevold M. Integrative nursing research — an important strategy to further the development of nursing science and nursing practice. Journal of Advanced Nursing. 1997; 25 (5):977–984. [ PubMed : 9147203 ]
• Kitchenham B., Charters S. ebse Technical Report Version 2.3. Keele & Durham. uk : Keele University & University of Durham; 2007. Guidelines for performing systematic literature reviews in software engineering.
• Kitsiou S., Paré G., Jaana M. Systematic reviews and meta-analyses of home telemonitoring interventions for patients with chronic diseases: a critical assessment of their methodological quality. Journal of Medical Internet Research. 2013; 15 (7):e150. [ PMC free article : PMC3785977 ] [ PubMed : 23880072 ]
• Kitsiou S., Paré G., Jaana M. Effects of home telemonitoring interventions on patients with chronic heart failure: an overview of systematic reviews. Journal of Medical Internet Research. 2015; 17 (3):e63. [ PMC free article : PMC4376138 ] [ PubMed : 25768664 ]
• Levac D., Colquhoun H., O’Brien K. K. Scoping studies: advancing the methodology. Implementation Science. 2010; 5 (1):69. [ PMC free article : PMC2954944 ] [ PubMed : 20854677 ]
• Levy Y., Ellis T.J. A systems approach to conduct an effective literature review in support of information systems research. Informing Science. 2006; 9 :181–211.
• Liberati A., Altman D. G., Tetzlaff J., Mulrow C., Gøtzsche P. C., Ioannidis J. P. A. et al. Moher D. The prisma statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. Annals of Internal Medicine. 2009; 151 (4):W-65. [ PubMed : 19622512 ]
• Lyden J. R., Zickmund S. L., Bhargava T. D., Bryce C. L., Conroy M. B., Fischer G. S. et al. McTigue K. M. Implementing health information technology in a patient-centered manner: Patient experiences with an online evidence-based lifestyle intervention. Journal for Healthcare Quality. 2013; 35 (5):47–57. [ PubMed : 24004039 ]
• Mickan S., Atherton H., Roberts N. W., Heneghan C., Tilson J.K. Use of handheld computers in clinical practice: a systematic review. bmc Medical Informatics and Decision Making. 2014; 14 :56. [ PMC free article : PMC4099138 ] [ PubMed : 24998515 ]
• Moher D. The problem of duplicate systematic reviews. British Medical Journal. 2013; 347 (5040) [ PubMed : 23945367 ] [ CrossRef ]
• Montori V. M., Wilczynski N. L., Morgan D., Haynes R. B., Hedges T. Systematic reviews: a cross-sectional study of location and citation counts. bmc Medicine. 2003; 1 :2. [ PMC free article : PMC281591 ] [ PubMed : 14633274 ]
• Mulrow C. D. The medical review article: state of the science. Annals of Internal Medicine. 1987; 106 (3):485–488. [ PubMed : 3813259 ] [ CrossRef ]
• Evidence-based information systems: A decade later. Proceedings of the European Conference on Information Systems ; 2011. Retrieved from http://aisel ​.aisnet.org/cgi/viewcontent ​.cgi?article ​=1221&context ​=ecis2011 .
• Okoli C., Schabram K. A guide to conducting a systematic literature review of information systems research. ssrn Electronic Journal. 2010
• Otte-Trojel T., de Bont A., Rundall T. G., van de Klundert J. How outcomes are achieved through patient portals: a realist review. Journal of American Medical Informatics Association. 2014; 21 (4):751–757. [ PMC free article : PMC4078283 ] [ PubMed : 24503882 ]
• Paré G., Trudel M.-C., Jaana M., Kitsiou S. Synthesizing information systems knowledge: A typology of literature reviews. Information & Management. 2015; 52 (2):183–199.
• Patsopoulos N. A., Analatos A. A., Ioannidis J.P. A. Relative citation impact of various study designs in the health sciences. Journal of the American Medical Association. 2005; 293 (19):2362–2366. [ PubMed : 15900006 ]
• Paul M. M., Greene C. M., Newton-Dame R., Thorpe L. E., Perlman S. E., McVeigh K. H., Gourevitch M.N. The state of population health surveillance using electronic health records: A narrative review. Population Health Management. 2015; 18 (3):209–216. [ PubMed : 25608033 ]
• Pawson R. Evidence-based policy: a realist perspective. London: SAGE Publications; 2006.
• Pawson R., Greenhalgh T., Harvey G., Walshe K. Realist review—a new method of systematic review designed for complex policy interventions. Journal of Health Services Research & Policy. 2005; 10 (Suppl 1):21–34. [ PubMed : 16053581 ]
• Petersen K., Vakkalanka S., Kuzniarz L. Guidelines for conducting systematic mapping studies in software engineering: An update. Information and Software Technology. 2015; 64 :1–18.
• Petticrew M., Roberts H. Systematic reviews in the social sciences: A practical guide. Malden, ma : Blackwell Publishing Co; 2006.
• Rousseau D. M., Manning J., Denyer D. Evidence in management and organizational science: Assembling the field’s full weight of scientific knowledge through syntheses. The Academy of Management Annals. 2008; 2 (1):475–515.
• Rowe F. What literature review is not: diversity, boundaries and recommendations. European Journal of Information Systems. 2014; 23 (3):241–255.
• Shea B. J., Hamel C., Wells G. A., Bouter L. M., Kristjansson E., Grimshaw J. et al. Boers M. amstar is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. Journal of Clinical Epidemiology. 2009; 62 (10):1013–1020. [ PubMed : 19230606 ]
• Shepperd S., Lewin S., Straus S., Clarke M., Eccles M. P., Fitzpatrick R. et al. Sheikh A. Can we systematically review studies that evaluate complex interventions? PLoS Medicine. 2009; 6 (8):e1000086. [ PMC free article : PMC2717209 ] [ PubMed : 19668360 ]
• Silva B. M., Rodrigues J. J., de la Torre Díez I., López-Coronado M., Saleem K. Mobile-health: A review of current state in 2015. Journal of Biomedical Informatics. 2015; 56 :265–272. [ PubMed : 26071682 ]
• Smith V., Devane D., Begley C., Clarke M. Methodology in conducting a systematic review of systematic reviews of healthcare interventions. bmc Medical Research Methodology. 2011; 11 (1):15. [ PMC free article : PMC3039637 ] [ PubMed : 21291558 ]
• Sylvester A., Tate M., Johnstone D. Beyond synthesis: re-presenting heterogeneous research literature. Behaviour & Information Technology. 2013; 32 (12):1199–1215.
• Templier M., Paré G. A framework for guiding and evaluating literature reviews. Communications of the Association for Information Systems. 2015; 37 (6):112–137.
• Thomas J., Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. bmc Medical Research Methodology. 2008; 8 (1):45. [ PMC free article : PMC2478656 ] [ PubMed : 18616818 ]
• Reconstructing the giant: on the importance of rigour in documenting the literature search process. Paper presented at the Proceedings of the 17th European Conference on Information Systems ( ecis 2009); Verona, Italy. 2009.
• Webster J., Watson R.T. Analyzing the past to prepare for the future: Writing a literature review. Management Information Systems Quarterly. 2002; 26 (2):11.
• Whitlock E. P., Lin J. S., Chou R., Shekelle P., Robinson K.A. Using existing systematic reviews in complex systematic reviews. Annals of Internal Medicine. 2008; 148 (10):776–782. [ PubMed : 18490690 ]

This publication is licensed under a Creative Commons License, Attribution-Noncommercial 4.0 International License (CC BY-NC 4.0): see https://creativecommons.org/licenses/by-nc/4.0/

• Cite this Page Paré G, Kitsiou S. Chapter 9 Methods for Literature Reviews. In: Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.
• PDF version of this title (4.5M)
• Disable Glossary Links

In this Page

• Introduction
• Overview of the Literature Review Process and Steps
• Types of Review Articles and Brief Illustrations
• Concluding Remarks

Related information

• PMC PubMed Central citations
• PubMed Links to PubMed

Recent Activity

• Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Ev... Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Evidence-based Approach

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Connect with NLM

National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894

Web Policies FOIA HHS Vulnerability Disclosure

Help Accessibility Careers

Literature Review: Outline, Strategies, and Examples [2023]

Writing a literature review might be easier than you think. You should understand its basic rules, and that’s it! This article is just about that.

Why is the literature review important? What are its types? We will uncover these and other possible questions.

Whether you are an experienced researcher or a student, this article will come in handy. Keep reading!

What Is a Literature Review?

• Step-by-Step Strategy

Let’s start with the literature review definition.

Literature review outlooks the existing sources on a given topic. Its primary goal is to provide an overall picture of the study object. It clears up the context and showcases the analysis of the paper’s theoretical methodology.

In case you want to see the examples of this type of work, check out our collection of free student essays .

Importance of Literature Review

In most cases, you need to write a literature review as a part of an academic project. Those can be dissertations , theses, or research papers.

Why is it important?

Imagine your final research as a 100% bar. Let’s recall Pareto law: 20% of efforts make 80% of the result. In our case, 20% is preparing a literature review. Writing itself is less important than an in-depth analysis of current literature. Do you want to avoid possible frustration in academic writing? Make a confident start with a literature review.

Sure, it’s impossible to find a topic that hasn’t been discussed or cited. That is why we cannot but use the works of other authors. You don’t have to agree with them. Discuss, criticize, analyze, and debate.

So, the purpose of the literature review is to give the knowledge foundation for the topic and establish its understanding. Abstracting from personal opinions and judgments is a crucial attribute.

Types of Literature Review

You can reach the purpose we have discussed above in several ways, which means there are several types of literature review.

What sets them apart?

In short, it’s their research methods and structure. Let’s break down each type:

• Meta-analysis implies the deductive approach. At first, you gather several related research papers. Then, you carry out its statistical analysis. As a result, you answer a formulated question.
• Meta-synthesis goes along with the inductive approach. It bases qualitative data assessment.
• Theoretical literature review implies gathering theories. Those theories apply to studied ideas or concepts. Links between theories become more explicit and clear. Why is it useful? It confirms that the theoretical framework is valid. On top of that, it assists in new hypothesis-making.
• Argumentative literature review starts with a problem statement. Then, you select and study the topic-related literature to confirm or deny the stated question. There is one sufficient problem in this type, by the way. The author may write the text with a grain of bias.
• Narrative literature review focuses on literature mismatches. It indicates possible gaps and concludes the body of literature. The primary step here is stating a focused research question. Another name for this type — a traditional literature review.
• Integrative literature review drives scientific novelty. It generates new statements around the existing research. The primary tool for that is secondary data . The thing you need is to review and criticize it. When is the best option to write an integrative literature review? It’s when you lack primary data analysis.

Remember: before writing a literature review, specify its type . Another step you should take is to argue your choice. Make sure it fits the research framework. It will save your time as you won’t need to figure out fitting strategies and methods.

Annotated Bibliography vs. Literature Review

Some would ask: isn’t what you are writing about is just an annotated bibliography ? Sure, both annotated bibliography and literature review list the research topic-related sources. But no more than that. Such contextual attributes as goal, structure, and components differ a lot.

For a more visual illustration of its difference, we made a table:

To sum up: an annotated bibliography is more referral. It does not require reading all the sources in the list. On the contrary, you won’t reach the literature review purpose without examining all the sources cited.

Literature Review: Step-by-Step Strategy

Now it’s time for a step-by-step guide. We are getting closer to a perfect literature review!

✔️ Step 1. Select the Topic

Selecting a topic requires looking from two perspectives. They are the following:

• Stand-alone paper . Choose an engaging topic and state a central problem. Then, investigate the trusted literature sources in scholarly databases.
• Part of a dissertation or thesis . In this case, you should dig around the thesis topic, research objectives, and purpose.

Regardless of the situation, you should not just list several literature items. On the contrary, build a decent logical connection and analysis. Only that way, you’ll answer the research question .

✔️ Step 2. Identify the Review Scope

One more essential thing to do is to define the research boundaries: don’t make them too broad or too narrow.

Push back on the chosen topic and define the number and level of comprehensiveness of your paper. Define the historical period as well. After that, select a pool of credible sources for further synthesis and analysis.

✔️ Step 3. Work with Sources

Investigate each chosen source. Note each important insight you come across. Learn how to cite a literature review to avoid plagiarism.

✔️ Step 4. Write a Literature Review Outline

No matter what the writing purpose is: research, informative, promotional, etc. The power of your future text is in the proper planning. If you start with a well-defined structure, there’s a much higher chance that you’ll reach exceptional results.

✔️ Step 5. Review the Literature

Once you’ve outlined your literature review, you’re ready for a writing part. While writing, try to be selective, thinking critically, and don’t forget to stay to the point. In the end, make a compelling literature review conclusion.

On top of the above five steps, explore some other working tips to make your literature review as informative as possible.

Literature Review Outline

We’ve already discussed the importance of a literature review outline. Now, it’s time to understand how to create it.

An outline for literature review has a bit different structure comparing with other types of paper works. It includes:

• Selected topic
• Research question
• Related research question trends and prospects
• Research methods
• Expected research results
• Overview of literature core areas
• Research problem consideration through the prism of this piece of literature
• Methods, controversial points, gaps
• Cumulative list of arguments around the research question
• Links to existing literature and a place of your paper in the existing system of knowledge.

It can be a plus if you clarify the applicability of your literature review in further research.

Once you outline your literature review, you can slightly shorten your writing path. Let’s move on to actual samples of literature review.

Literature Review Examples

How does a well-prepared literature review look like? Check these three StudyCorgi samples to understand. Follow the table:

Take your time and read literature review examples to solidify knowledge and sharpen your skills. You’ll get a more definite picture of the literature review length, methods, and topics.

Do you still have any questions? Don’t hesitate to contact us! Our writing experts are ready to help you with your paper on time.

❓ What Is the Purpose of a Literature Review?

Literature review solves several problems at once. Its purpose is to identify and gather the top insights, gaps, and answers to research questions. Those help to get a general idea of the degree of topic exploration. As a result, it forms a basis for further research. Or vice versa: it reveals a lack of need for additional studies.

❓ How Do You Structure a Literature Review?

Like any other academic paper, a literature review consists of three parts: introduction, main body, and the conclusion. Each of them needs full disclosure and logical interconnection

The introduction contains the topic overview, its problematics, research methods, and other general attributes of academic papers.

The body reveals how each of the selected literature sources answers the formulated questions from the introduction.

The conclusion summarizes the key findings from the body, connects the research to existing studies, and outlines the need for further investigation.

To ensure the success of your analysis, you should equally uphold all of these parts.

❓ What Must a Literature Review Include?

A basic literature review includes the introduction with the research topic definition, its arguments, and problems. Then, it has a synthesis of the picked pieces of literature. It may describe the possible gaps and contradictions in existing research. The practical relevance and contribution to new studies are also welcome.

❓ What Are the 5 C’s of Writing a Literature Review?

Don’t forget about these five C’s to make things easier in writing a literature review:

Cite . Make a list of references for research you’ve used and apply proper citation rules. Use Google Scholar for this.

Compare . Make a comparison of such literature attributes as theories, insights, trends, arguments, etc. It’s better to use tables or diagrams to make your content visual.

Contrast . Use listings to categorize particular approaches, themes, and so on.

Critique . Critical thinking is a must in any scientific research. Don’t take individual formulations as truth. Explore controversial points of view.

Connect . Find a place of your research between existing studies. Propose new possible areas to dig further.

❓ How Long Should a Literature Review Be?

In most cases, professors or educational establishment guidelines determine the length of a literature review. Study them and stick to their requirements, so you don’t get it wrong.

If there are no specific rules, make sure it is no more than 30% of the whole research paper.

If your literature review is not a part of the thesis and goes as a stand-alone paper — be concise but explore the research area in-depth.

• Literature Reviews – The University of North Carolina at Chapel Hill
• What is a literature review? – The Royal Literary Fund
• Literature Review: Purpose of a Literature Review – University of South Carolina
• The Literature Review: A Few Tips On Conducting It – University of Toronto
• Steps for Conducting a Lit Review – Florida A&M University Libraries
• Types of Literature Review – Business Research Methodology
• How to Conduct a Literature Review: Types of Literature Reviews – University Libraries
• Annotated bibliography VS. Literature Review – UNT Dallas Learning Commons
• Literature Review: Conducting & Writing – University of West Florida
• Share to Facebook
• Share to Twitter X
• Share to LinkedIn

You might also like

105 literature review topics + how-to guide [2023], study guide on the epic of gilgamesh, essay topics & sample, the necklace: summary, themes, and a short story analysis.

• Open access
• Published: 20 May 2023

Measurement of student engagement in health professions education: a review of literature

• Salah Eldin Kassab   ORCID: orcid.org/0000-0003-1482-1996 1 , 2 ,
• Mohamed Al-Eraky   ORCID: orcid.org/0000-0003-2015-7630 1 ,
• Walid El-Sayed   ORCID: orcid.org/0000-0002-6647-516X 1 , 3 ,
• Hossam Hamdy   ORCID: orcid.org/0000-0001-6096-6139 1 &
• Henk Schmidt   ORCID: orcid.org/0000-0001-8706-0978 1 , 4  

BMC Medical Education volume  23 , Article number:  354 ( 2023 ) Cite this article

2795 Accesses

1 Altmetric

Metrics details

Student engagement is a complex multidimensional construct that has attained great interest in health professions education (HPE). Definition and conceptualization of student engagement is an important step that should drive the development of the instruments for its measurement. We have recently proposed a comprehensive framework for student engagement in HPE with a definition of engagement as student investment of time and energy in academic and non-academic experiences that include learning, teaching, research, governance, and community activities. The dimensions of student engagement in this framework included the cognitive, affective, behavioral, agentic, and socio-cultural. Guided by the student engagement framework, this non-systematic review aims to identify, critically appraise, and summarize the existing methods for measuring student engagement in HPE. Extrapolating from higher education literature, we attempted to link the theoretical perspectives of student engagement with the published methods of its measurement in HPE context. In addition, we have described the different methods of measuring student engagement including self-report surveys, real time measures, direct observation, interviews/focus groups, and the use of multiple instruments. The span of engagement dimensions measured by self-report surveys ranges from one to five dimensions. However, measurement of agentic and sociocultural dimensions of engagement in HPE is still limited and further research is required. We have also reflected on the existing methods of measuring engagement of students as active partners in HPE. The review also describes the advantages, limitations, and psychometric properties of each method for measuring student engagement. We ended the review with a guiding conclusion on how to develop and select an instrument for measuring student engagement in HPE. Finally, we addressed the gaps in the literature about measuring engagement of HPE students and future research plans.

Peer Review reports

The research agenda on student engagement in education has witnessed a progressive rise during the past three decades. The main drive for this rise is the significance of student engagement as a predictor of academic success, well-being, satisfaction, increased retention, decreased burnout, and enhanced self-directed learning [ 1 ]. Furthermore, engagement of students in learning enhances teacher motivation [ 2 ]. Accordingly, engagement of students has been used as an indicator for the quality of medical programs [ 3 ] and a measure of institutional excellence in medical education [ 4 ]. We have also recently reviewed the different aspects related to this important construct and its implications on health professions education [ 1 , 5 ]. Despite the presence of several instruments in the literature for measuring engagement of HPE students, there are no currently existing comprehensive reviews that describe these methods. There are also no guiding principles on how to develop and select an instrument for measuring student engagement in HPE.

Definition of student engagement

Previous literature in higher education has included several definitions of student engagement according to the underlying theoretical perspectives. The prevailing three theoretical underpinnings that explain student engagement include the psychological, behavioral, and psychosocial perspectives [ 1 , 6 ]. The psychological perspective considers engagement as an internal psychological state of students. According to this perspective, student engagement is defined as the students’ psychological state of activity that makes them feel activated, exert effort, and be absorbed during learning activities and students’ state of connection with the school community [ 7 ]. The behavioral perspective explains engagement as both the student behavior and the institutional factors that drive the student engagement. Accordingly, student engagement from this perspective is defined as the time and effort students dedicate to educationally purposeful activities and the practices that institutions apply to motivate students to participate in these activities [ 8 ]. The sociocultural perspective addresses the role of social, cultural, and political factors in student engagement. Accordingly, sociocultural engagement is defined as the student ability of expanding viewpoints and providing awareness of, and appreciation for, others from diverse social and cultural backgrounds [ 9 ].

Student engagement is multidimensional and multi-level

The multidimensional nature of student engagement as a construct poses a practical difficulty to its measurement. Student engagement is conceptualized into behavioral, cognitive, emotional, agentic, and sociocultural dimensions that are measured by relevant indicators. Behavioral engagement can be measured by indicators such as student attendance, participation in curricular or extracurricular activities, effort, and ability to persevere in academic pursuits despite challenges. Indicators of emotional engagement include the emotions students experience towards their learning, peers, faculty, and school. These emotions include happiness, enthusiasm, pride, enjoyment, and feeling of bonding. Cognitive engagement includes absorption in learning, metacognition, perceived value of academic tasks and use of high-order cognitive skills. Agentic engagement indicates the student power to influence to their education, their future lives, and their social environment [ 10 ]. Indicators of agentic engagement inside the classroom include the active contribution of students to their learning process [ 11 ]. Agentic engagement outside the classroom can be measured by the student involvement in teaching of their peers, active participation in school governance, and involvement in community activities. Sociocultural engagement refers to the extent of students’ awareness of, and appreciation for, the diverse perspectives and experiences represented in their learning community. Indicators of sociocultural engagement include appreciation for different cultural backgrounds, willingness to engage in cross-cultural dialogue, and accepting to learn from others from different perspectives [ 9 ].

Student engagement can be molded according to the changes in the surrounding environment. These changes will have direct implications on the methods used for its measurement. For example, student engagement varies according to the type of learning activity (e.g., large classroom, small group learning, self-learning). Similarly, engagement of students differs according to the time scale, which can range from an engagement in a short learning activity to engagement along the duration of a course or a program.

Spheres of student engagement

The spheres of students’ engagement include either engagement in their own learning or engagement as partners in education. The areas of engagement as partners include provision of education, scholarly research, governance, and community activities [ 4 ]. Accordingly, student engagement has been defined as academic experiences of students in learning, teaching, and research, at the cognitive, behavioral, and emotional levels through interactions with peers, faculty, and college community [ 12 ]. We have recently provided a comprehensive definition of student engagement in HPE as the student investment of time and energy in academic and non-academic experiences that include learning, teaching, research, governance, and community activities. Students are involved in these aspects at the cognitive, affective, behavioral, agentic, and socio-cultural dimensions [ 1 ].

Student engagement in technology-enhanced learning (TEL) environments

Technology-enhanced learning (TEL) environments are essentially conducted online and offer both opportunities and challenges for student engagement. These environments offer more flexibility and autonomy for students to customize their learning experience according to the most suitable time, method, and place. However, TEL requires students to possess the technology literacy that allows them to navigate online platforms, manage multimedia resources, and manage digital information. Furthermore, students require adaptation to different methods of communication and social skills compared with face-to-face settings to get engaged in online learning activities. Therefore, methods of measuring student engagement in face-to-face learning environments may not be suitable for use in online environments. For example, direct observation is suitable for measuring behavioral engagement of students through direct interactions in face-to-face classrooms. However, measuring behavioral engagement of students in an online environment may need to rely on other measures such as self-report surveys, data analytics and log files.

This review aims to provide an overview of the methods of measuring student engagement in HPE. We linked each method with the underpinning theoretical perspectives, and described the measured engagement dimensions, advantages and limitations, and the psychometric properties of each method. In addition, we addressed the current gaps in the literature about measuring student engagement in HPE and directions for future research.

Methodology

This manuscript represents a non-systematic literature review employing an explicit search strategy to reduce the biases in selection of included articles. The review included articles published in English with the focus of research on conceptualization and measurement of student engagement, and the main subjects are HPE students. We conducted literature search using the following databases: MEDLINE, PubMed, ProQuest, SCOPUS, Education Resources Information Centre (ERIC), Science Direct, and EBESCO. We searched for peer-reviewed research articles in the databases by title and abstract using key terms such as student engagement, engagement, learner engagement, students as partners, and partnership. We combined the previous terms with other words such as health professions, medical, dentistry, pharmacy, nursing, allied health professions, clinical psychology, physical therapy, nutrition, and occupational therapy. In addition, we selected relevant articles from the references list of identified key publications in student engagement. The literature search was limited to articles between 1990 and November 2022. Searching the databases yielded 3019 articles, while an additional 617 articles were identified through searching the listed references and hand-searching of HPE journals. Following deduplication and excluding irrelevant articles through screening of titles and abstracts, 144 articles were selected. Individual screening of full text articles by the two authors (SK and WE) led to the selection of 71 articles to be included in the review with a date of publication ranging from 2003 to 2022. Although the main emphasis of the review is on studies relevant to HPE context, authors agreed to include additional relevant articles ( n  = 19) related to non-HPE contexts. Articles that used self-reports with only one statement for measuring student engagement were excluded from the review. We used EndNote (version 7, Clarivate Analytics, Philadelphia, United States) as the reference manager software for the bibliography of the identified articles.

Methods of measuring student engagement in HPE

The outcome of the literature search resulted in different measures of student engagement in HPE. The common methods of measuring student engagement are self-report surveys, real-time measures, direct observation, interviews, or a combination of more than one method (Fig.  1 ).

Frequency of the methods for measuring student engagement in health professions education

Self-report surveys

Self-report surveys are the most used methods for measuring student engagement in HPE, representing approximately two thirds of the reported methods. These surveys are easy to administer, cheap, and can sample many students in a brief period. Furthermore, self-reports can measure unobservable aspects of engagement such as cognitive and emotional dimensions [ 13 ]. The number of dimensions measured by the self-reports varies from one to five dimensions. However, the problem with self-reports is the inability to measure the dynamic nature of student engagement in certain learning situations. One attempt to overcome this shortcoming is the experience sampling using short questionnaires that are distributed several times during a learning activity [ 14 ]. Most surveys are not able to capture the complex nature of the student engagement construct. Even some of the questionnaires that cover the multiple dimensions of engagement lack the alignment between conceptualization and the method of measurement. Below is a detailed description of the common self-report measures in HPE literature and a summary of these instruments in provided in Table 1 .

Self-reports measuring one engagement dimension

Situational cognitive engagement questionnaire

This questionnaire measures cognitive engagement defined as a psychological state in which students exert a significant amount of effort to understand the topic at hand and in which they persist studying over a prolonged period [ 15 ]. Items are scored on a 5-point Likert scale, ranging from 1 (not true at all) to 5 (very true for me). The validity of the questionnaire has been tested in a large sample of applied science students with model fit statistics supporting the model [ 15 ]. Furthermore, coefficient H was 0.93 and 0.88 when evaluated on students in applied sciences and medicine [ 14 , 15 ], respectively. Because of the short nature of this questionnaire, it allows multiple measurements of cognitive engagement in response to contextual changes. However, the limited scope of this questionnaire to the cognitive dimension does not allow measurement of other relevant engagement dimensions during collaborative learning in PBL tutorials such as emotional, behavioral, and social dimensions. Another limitation of the questionnaire is the use of behavioral engagement indicators such as effort and persistence as measures of cognitive engagement.

Learners' engagement and motivation questionnaire

This questionnaire has been used for measuring cognitive engagement of health professions education students in multimedia learning [ 16 ]. The questionnaire is based on conceptualizing student engagement as a state of flow (absorption, full concentration, intense enjoyment, and distortion of time awareness). The instrument consists of six items measured on a 7-point Likert scale where 1= not at all true of me to 7=very true of me. Cognitive engagement is conceptualized into three subconstructs: 1) Attention focus (2 items), 2) Intrinsic interest (2 items), and 3) Curiosity (2 items). Although the instrument is declared for measuring cognitive engagement, indicators were a mix of behavioral (attention) and emotional (interest and curiosity) engagement. However, the scale has been pre-validated for internal structure and exploratory factor analysis demonstrated a unidimensional factor structure. Internal consistency reliability using Cronbach alpha was 0.93 to 0.95.

Self-reports measuring two engagement dimensions

Classroom engagement survey (CES)

The CES was initially designed in general education settings and imported for use in nursing education [ 33 ]. The CES consists of 8 items designed to measure behavioral and emotional engagement of students [ 33 ]. Behavioral engagement is measured by student participation in the classroom (five items) and emotional engagement measures their enjoyment (three items) . Another version of the CES consists of 8 items representing the behavioral (3 items), emotional (3 items), and cognitive (2 items) dimensions [ 34 , 35 ]. However, a major limitation of this version is the lack of validity support for the multidimensionality of the construct. In addition, two items are measuring behavioral and cognitive engagement at the group level rather than individual students. Items are scored on a five-point Likert scale (1, strongly disagree; 2, disagree; 3, neither agree nor disagree; 4, agree; 5, strongly agree). The summed scores range from 5 to 40. A higher score indicates greater engagement of students with a score of 24 considered as a neutral score. The use of CES in HPE demonstrated an internal consistency reliability of the questionnaire ranged from Cronbach’s alpha coefficient range from 0.83 [ 34 ] to 0.88 [ 33 , 35 ].

User Experience Questionnaire (UEQ)

The long version of the UEQ (68 items) has been developed and validated for measuring the experience in immersive virtual environments using participants mainly from information and communication technology or computer sciences [ 49 ]. The shorter version of the questionnaire has been adapted for use for measuring engagement of medical students during the use of 360 O videos in Anatomy education [ 18 ]. This version consists of 8 items that measure student engagement as a state of flow characterized by immersion (2 items), enjoyment (2 items), loss of time awareness (2 items), and overall involvement (2 items). Students are asked to score their degree of agreement with each statement on a scale of 0–100, and the average score represents the degree of student engagement. The main limitation of the UEQ is the lack of evidence for the construct validity either for its internal structure using factor analysis or criterion-related evidence by testing relationships to other variables.

User Engagement Scale-20 (UES-20)

The UES-20 has been developed for measuring engagement of commercial users with the learning resources to which they are exposed in online environments [ 50 ]. The scale was then imported for measuring engagement of medical students during their e-learning for diagnostic imaging using adaptive tutorials [ 17 ]. Adaptive tutorials are intelligent online tutoring systems, which provide a personalized learning experience of students through immediate feedback that is modified according to individual student responses [ 17 ]. The instrument measures the cognitive and emotional dimensions of engagement with e-learning resources. The scale consists of 20 items scored on a 5-point Likert scale (1 = strongly disagree and 5 = strongly agree) clustered under four subscales. The subscales include focused attention (ability to concentrate and absorb information), perceived usability (affective and cognitive responses to the resource), novelty and involvement (level of triggered interest, feeling of immersion and having fun), and aesthetic appeal (impression made by the visual appearance of the user interface). Factor analysis demonstrated a four-factor structure, which supports the multidimensionality of the questionnaire [ 50 ] [ 17 ].

Self-reports measuring three engagement dimensions

Utrecht Work Engagement Scale (UWES)

This scale is one of the most used self-reports in HPE and has an established theoretical basis on the psychological perspective of engagement and the schoolwork engagement model. According to this model, engagement is conceptualized as a positive state of study-related fulfillment characterized by vigor (emotional), dedication (cognitive), and absorption (behavioral) [ 51 ]. Different versions of the scale have been used in the HPE literature with larger versions consisting of 17 items [ 19 , 20 ], 15 items [ 21 , 22 ], and 14 items [ 23 , 24 , 25 ]. Shorter versions that consist of 9 items [ 26 , 27 , 28 ] and even 3 items [ 29 ] are also used. For each of the items, students are asked to identify their level of engagement using a seven-point Likert scale (1 = never to 7 = always). The sum scores of items are divided by the number of items in the scale to represent total engagement score. The different versions of the scale have demonstrated good psychometric properties in various contexts as evidenced by good to excellent internal consistency reliability and factor analysis findings that support the theoretical model [ 19 , 23 , 25 ]. Other sources of validity evidence for the questionnaire are the negative correlations between student engagement scores using UWES and perceived stress [ 21 , 25 ] and burnout [ 21 , 24 , 28 ]. In addition, there is significant positive association between perceived satisfaction of basic psychological needs (autonomy, competence, and relatedness) and student engagement [ 28 ]. Furthermore, student engagement is promoted by students’ academic self-efficacy beliefs, students’ active self-care, and resilience [ 27 ].

University Student Engagement Inventory (USEI)

The earlier version of the USEI was originally developed in Portuguese to measure student engagement [ 52 ]. This version is composed of 32 items distributed in three dimensions covering behavioral (11 items), emotional (10 items), and cognitive (11 items) engagement. However, a reduced version that consists of 15 items has demonstrated better psychometric properties [ 30 ]. The short version has been used for measuring engagement of medical [ 31 ], dental [ 30 ], and pharmacy [ 32 ] students. Questionnaire items are distributed in a three-factor structure covering the three dimensions of engagement: behavioral (5 items), emotional (5 items), and cognitive (5 items). Items are assessed on a 5-point scale ranging from 1 (never) to 5 (always). An evidence of construct validity of the scores from the reduced version has been demonstrated by confirmatory factor analysis, which supported the three-factor model with significant correlations between the subscales [ 30 , 31 ]. In addition, emotional and behavioral engagement scores correlated negatively with perceived burnout [ 32 ]. The questionnaire has also demonstrated an acceptable reliability (≥ 0.7) using composite reliability (CR) and Cronbach’s alpha [ 30 ].

Technology-enhanced Learning (TEL) engagement scale

This scale intends to measure the engagement of students in TEL resources, with an example of its application in Anatomy education [ 36 ]. The instrument consists of 19 items scored on a five-point Likert scale (1 = strongly disagree and 5 = strongly agree). Exploratory factor analysis yielded a three-factor structure as follows: satisfaction (8 items), goal setting and planning (7 items), and physical interaction (4 items). These emerging factors generally conform with emotional, cognitive, and behavioral dimensions of engagement, respectively. In addition, there are significant correlations between the three engagement dimensions. TEL engagement score is calculated by summing the responses from each item with the minimum score of 19 and a maximum score of 95. The reported internal consistency reliability of the scale was 0.86 with acceptable reliability of each of the three engagement dimensions. Although the scale has proved evidence of validity in Anatomy learning resources, the outcome of its application in other HPE subjects is unclear.

Self-reports measuring four engagement dimensions

Online Student Engagement Scale (OSE)

This scale attempts to measure the behavior, thoughts, and feelings of students in online learning. The initial version of the scale has been adapted from the Student Course Engagement Questionnaire (SCEQ) for use in online communication engineering courses [ 37 ]. The questionnaire was then imported for use in nursing education [ 37 , 38 , 39 ]. It comprised 19 items divided into four factors: 1) skills engagement (cognitive), 2) emotional engagement, 3) participation engagement (behavioral), and 4) performance engagement [ 38 , 40 ]. The items are scored on a 5-point Likert scale using the following response categories: very characteristic of me (5), characteristic of me (4), moderately characteristic of me (3), not really characteristic of me (2) or not at all characteristic of me (1). The total engagement score represents the sum for the four engagement dimensions with ninety-five considered as the maximum score. The scale demonstrates high internal consistency reliability with a Cronbach alpha ranging from 0.91 [ 37 , 38 ] to 0.95 [ 39 ]. In addition, factor analysis yielded a four-factor structure including skills, emotional, participation, and performance [ 37 ]. The main disadvantage of this questionnaire is mixing between the dimensions and outcomes of engagement (performance engagement).

College students’ learning engagement scale in cyberspace

This scale has been used for measuring learning engagement in online courses for Chinese nursing students [ 41 ]. Authors conceptualized online engagement into four dimensions: behavioral, emotional, cognitive, and interactive engagement. The scale consists of 19 items rated on a 5-points Likert scale ranging from 1 (completely inconsistent) to 5 (completely consistent), with a total score of 19 to 95. A higher total score indicated higher online learning engagement. Confirmatory Factor Analysis (CFA) showed that the structure of this scale was stable and the Cronbach's Alpha coefficient was 0.972.

Self-reports measuring student engagement as partners

Educational student engagement scale

This questionnaire measures the engagement of medical students as active partners in the provision of education [ 48 ]. The questionnaire consists of six items designed to be in line with the ASPIRE criteria for institutional excellence in student engagement [ 4 , 53 ]. The items cover mainly the agentic engagement dimension and include student role in curriculum evaluation, peer teaching, self- and peer assessment, and the role of their feedback on curriculum development. Students are asked to indicate their level of agreement on each item based on a five-point Likert scale where 1= strongly disagree and 5 = strongly agree. The average score of the six items represents the mean level of student engagement. The internal consistency reliability of the questionnaire is 0.88. The questionnaire has also demonstrated evidence of predictive validity by the positive relationships between student engagement scores and student learning outcomes. However, the scope of the questionnaire is limited to measuring engagement in provision of education and does not measure the other aspects of student engagement as partners such as engagement in governance, scholarly research, and community activities.

Measures of institutional level of student engagement

National Survey of Student Engagement (NSSE)

The NSSE originated in the early 20 th century as a tool for measuring student engagement at the institutional level to improve the undergraduate experiences of students, document effective institutional practices, and benchmark between higher education institutions [ 8 ]. This self-report survey measures student engagement on an annual basis and is designed based on the behavioral perspective of student engagement. The instrument consists of approximately 80 items with 10 indicators representing five themes of student engagement in addition to 6 high impact practices. The themes include academic challenge (4 indicators), learning with peers (2 indicators), experiences with faculty (2 indicators), and campus environment (2 indicators). The academic challenge theme covers higher-order learning, reflective & integrative learning, learning strategies, and quantitative reasoning. Learning with peers includes collaborative learning and discussions with diverse others. Experience with faculty includes student-faculty interaction and effective teaching practices. Campus environment includes quality of interactions and supportive environment. High impact practices include service-learning, learning community, research with faculty, internship or field experience, study abroad, and culminating senior experience. In the HPE research context, NSSE has been used for measuring engagement of nursing students [ 42 , 43 ]. The main limitation of the NSSE is mixing between indicators, drivers, and outcomes of engagement. One of the surveys developed by items borrowed from NSSE is the Survey of Student Engagement (SSE) instrument. The SSE instrument has been validated [ 44 ] and used for measuring engagement of medical students [ 45 , 46 , 47 ]. The instrument consists of 14 items grouped under three categories: 1) collaborative learning (4 items), 2) cognitive development (five items), and 3) personal skills (4 items). Items are scored on a 4-point Likert scale ranging from 1 = Never to 4 = very often, and the total engagement score is the sum of the scores in the three dimensions.

Classroom Survey of Student Engagement (CLASSE)

CLASSE is an adapted version of NSSE to measure engagement of students at the classroom level [ 54 ]. The purpose is to provide feedback to institutions on how to enhance instructional practices for better engagement of students. The CLASSE questionnaire consists of 39 items [ 55 ] with a shorter version of 19 items [ 56 ]. Each is assessed on a 4-point Likert scale (0 = never, 1= one time, 2= three to five times, and 3= more than five times) and then multiplied by 33.3 to produce a score ranging from 0 (lowest) to 100 (highest). The questionnaire consists of three dimensions: 1) active and collaborative learning, 2) student –faculty interaction, and 3) level of academic challenge. The questionnaire demonstrated good internal consistency reliability in HPE settings with a Cronbach’s alpha = 0.89 for the 39-item version [ 55 ].

Direct observation

Direct observation can measure the engagement of individual students or the whole classroom during a learning activity. The observable indicators for student engagement are mainly focused on the behavioral dimension such as attention, asking questions, and participating in classroom activities. Observation measures can provide real-time changes in student engagement and describe rich information about the contextual factors affecting engagement. However, they are time-consuming, require training of observers, and limited to measuring the behavioral dimension of engagement. In addition, direct observation mostly includes a small number of students, which limits the generalizability of these measures. In the HPE literature, there are two reported direct observation methods, which are STROBE and In-class engagement measure (IEM).

The name of the STROBE instrument refers to the strobe light that intermittently captures events at regular intervals. The instrument consists of 5-minute observational cycles repeated during the classroom activities. During each cycle, the trained observer records the behavior of the teacher and four students, and the process is repeated. Categories of student behaviors on the STROBE include: 1) Learner-to-learner engagement, such as speaking, listening, or both, 2) Learner-to-instructor engagement, such as speaking, listening, or both, and 3) Self-engagement: Learner reading, writing, or otherwise not visibly interacting with other learners or instructor. The observers also write free comments at the end of each 5-minute observation cycle. The reliability of the instrument measured by inter-observer agreement of observers who simultaneously scored the engagement of students was good to excellent [ 57 ]. In addition, validity-evidence was confirmed by the findings that student-student and student-instructor engagement were greater in PBL compared with traditional lectures [ 57 , 58 ], and that engagement scores using STROBE correlated with student self-report of engagement [ 57 ].

In-class engagement measure (IEM)

The instrument is a revised form of STROBE to determine the level of engagement of students and teachers in the classroom settings on direct observation. Each observation cycle includes recording the behaviors of an instructor and four randomly selected students as snapshots for 5-min cycles [ 59 , 60 ]. Each 5-min cycle consists of four 20-sec observations of individual learners. The observer scores student behavior on a scale 1 to 5 where 1 to 2 is non-participating personal without any communication and 3 to 5 is gradually increasing levels of participation and communication with the instructor and peers. The instrument demonstrated content-related validity evidence by review from education experts and criterion-related evidence of validity by the significantly higher engagement scores in active learning compared with traditional classes. In addition, the reliability of the instrument was proved by excellent inter-observer agreement in scores.

Real-time measures

Student engagement is a dynamic construct that responds to changes in the educational context. Therefore, several real-time measures have been developed to measure moment-to-moment changes in student engagement as it unfolds especially at the level of an educational activity. Examples of real-time measures applied for measuring student engagement in HPE are log files, physiological measures, and eye tracking. Log files are computer-generated data files that record system-related information including the internet usage patterns and activities. Log files can capture several indicators of behavioral engagement of students in technology-enhanced learning environments [ 61 , 62 , 63 , 64 , 65 , 66 , 67 , 68 , 69 ]. For example, a computer-generated engagement metric used multiple indicators to measure engagement of medical students with virtual patients such as time on page, MCQ answer accuracy, use of a clinical reasoning tool, and scores of students' written summary statements based on the VP encounter [ 69 ]. In addition, log files and physiological measures can automatically capture indicators of cognitive and emotional engagement (Table 2 ).

For example, heart rate changes are used for measuring cognitive engagement of medical students in different types of class activities [ 70 ]. Eye-tracking is another indicator of engagement with the assumption that fixating the eyes on text or images for longer period indicate that students are cognitively engaged with the subject [ 71 ]. For example, eye-tracking has been used for measuring engagement of medical students with moulage simulations [ 72 ]. Real-time measures have the advantage of collecting large amounts of information in a short period. The collected data from real-time measures are usually precise because they are not subject to human errors or bias. However, analyzing this large volume of data could be challenging. In addition, these measures could be expensive and difficult to use in real educational environments [ 73 ].

Interviews and focus groups

Interviews and focus groups have the advantage of collecting in-depth information about student engagement. The collected information is usually deep and rich as students have the chance to explain how their engagement unfolds in a learning environment. By discussing with HPE students, they can explain the contextual factors that trigger or inhibit their engagement [ 74 , 75 , 76 , 77 , 78 , 79 , 80 ]. Students can also explain the types and characteristics of their engagement [ 81 ] and how they get engaged in learning activities [ 79 ]. Despite these advantages, collecting and analyzing qualitative data from interviews and focus groups are time-consuming and require training of interviewers [ 13 ]. In addition, the small sample of students and interviewer biases could limit the generalizability of conclusions about student engagement.

e) Multiple methods

The comprehensive nature of the engagement construct made it almost impossible to measure all its components using a single instrument. To address this problem, several studies have used mixed quantitative and qualitative methods to triangulate the evidence about student engagement [ 72 , 82 , 83 , 84 , 85 , 86 , 87 ]. Studies have also used multiple quantitative measures to capture more dimensions of this complex construct [ 88 ]. Another purpose of using multiple methods is to identify the contextual factors that drive/inhibit student engagement as well as the outcomes of engagement. For example, a study used self-report surveys, real-time measures, and interviews for measuring engagement of medical students in Anatomy and Histology by using digital games [ 82 ]. In this study, measures were not only focused on indicators related to the engagement construct, but also on antecedents and outcomes of engagement [ 82 ]. Another study used direct observation, work sample analysis, teacher rating, and student self-report to investigate the role of virtual patient simulations (VPS) in fostering student engagement [ 83 ]. The methods used were a mix between measuring flow (a state of engagement) and antecedents of engagement such as motivation, interest, and relevance [ 83 ]. Another study used both log files and focus group discussion to examine how visual learning analytics tools such as learning dashboards can support medical students’ cognitive engagement in the classroom [ 85 ]. The log files were used for measuring cognitive engagement while the focus group discussion explored the perceptions of students about their cognitive engagement [ 85 ]. Furthermore, a study used Immersion Score Rating Instrument (ISRI), engagement self-report, eye-tracking, and stimulated recall interviews to explore how the moulage authenticity impacts on student engagement [ 72 ]. Although multiple methods can capture different aspects of engagement, the real challenge is how to reconcile the inconsistency in the findings from different methods and combine these findings to achieve a more comprehensive image of student engagement.

Conclusions and future directions

The measurement of student engagement in HPE has been a challenge despite the progressive interest of studying the construct for more than two decades. We conclude this review by highlighting important points to consider before developing and selecting instruments for measuring student engagement in HPE as shown in Table 3 .

First , conceptualizing and defining student engagement is an important step in its operationalization and developing the appropriate methods of measurement. Furthermore, aligning the methods of measuring engagement with the underlying theoretical perspectives would streamline the comparison of findings across student engagement studies. Most of the available methods focus on measuring the psychological perspective of student engagement, while the behavioral perspective is limited to measuring student engagement at the institutional level. However, methods of measuring engagement from the socio-cultural theoretical perspective in the HPE literature are still in infancy. This is particularly relevant to the sphere of student engagement as partners in HPE education where theories such as Community of Practice (CoP) and Positioning theory are applicable [ 1 ].

Second , an important consideration during the design or selection of an instrument for measuring student engagement is identifying the dimensions of engagement that can be measured. Several instruments are available in the HPE literature for measuring behavioral, cognitive, and emotional engagement of students. However, the instruments designed to measure the agentic and socio-cultural dimensions of student engagement in HPE are limited and should be the target for future research.

Third , a plethora of publications in HPE address the measurement of student engagement in the sphere of own learning while measuring engagement of students as partners has been limited. Furthermore, most of the existing instruments in HPE literature about measuring student engagement as partners are not focusing on the engagement construct. Instead, the instruments used in these studies are measuring the drivers [ 78 , 80 , 89 ] and outcomes [ 84 , 90 ] of student engagement. While the existing literature has provided valuable insights into the topic of student engagement as partners, further research is required to explore its practical applications and potential impact in HPE settings.

Fourth , student engagement has been conceptualized as a multi-level construct with variable time scales. Therefore, the granularity of measured engagement (learning activity, course, school, university) should be clearly identified at the outset of the study [ 7 ]. This issue has important implications on the appropriate method for each engagement level. For example, student engagement in a short learning activity can better be measured by direct observation or real-time measures. On the other hand, student engagement at the macro-level of the program can be measured by self-report surveys or interviews.

Finally , it is important to note that disengagement is not at the opposite end of the engagement spectrum. Engagement and disengagement are conceptually considered as distinct constructs with different outcomes. Accordingly, engagement and disengagement dimensions are distinct and require different structures of the instruments used for their measurement. Specifically, there is a lack of research on the measurement of student disengagement in HPE, highlighting the need for more studies to explore the different aspects related to this construct.

Availability of data and materials

Not applicable.

Abbreviations

Classroom engagement survey

Classroom Survey of Student Engagement

Community of Practice

Health professions education

In-class engagement measure

National Survey of Student Engagement

Online Student Engagement Scale

Technology-enhanced Learning

User Experience Questionnaire

User Engagement Scale-20

University Student Engagement Inventory

Utrecht Work Engagement Scale

Kassab SE, Taylor D, Hamdy H. Student engagement in health professions education: AMEE Guide No. 152. Med Teach. 2022;1–17.

Frenzel AC, Goetz T, Lüdtke O, Pekrun R, Sutton RE. Emotional transmission in the classroom: exploring the relationship between teacher and student enjoyment. J Educ Psychol. 2009;101(3):705–16.

Article   Google Scholar  

WFME. Basic Medical Education, WFME Global Standards for Quality Improvement. 2020 Available from: https://wfme.org/download/bme-standards-2020/http-wfme-org-wp-content-uploads-2017-05-wfme-bme-standards-2020-pdf/ .

Harden RM, Roberts TE. ASPIRE: international recognition of excellence in medical education. Lancet. 2015;385(9964):230.

Kassab SE, El-Sayed W, Hamdy H. Student engagement in undergraduate medical education: a scoping review. Med Educ. 2022;56(7):703–15.

Kahu ER. Framing student engagement in higher education. Stud High Educ. 2013;38(5):758–73.

Wong ZY, Liem GAD. Student Engagement: Current State of the Construct, Conceptual Refinement, and Future Research Directions. Educ Psychol Rev. 2021;34(1):107-38.

Kuh GD. The national survey of student engagement: Conceptual and empirical foundations. New Dir Inst Res. 2009;2009(141):5–20.

Google Scholar  

GQ F. Sociocultural and civic engagement.: Saratoga Springs, NY: Suny Empire State College; 2022 [Available from: https://www.esc.edu/global-learning-qualifications-framework/learning-domains/engagement/ .

Klemenčič M. From student engagement to student agency: conceptual considerations of European policies on student-centered learning in higher education. High Educ Pol. 2017;30(1):69–85.

Reeve J, Tseng C-M. Agency as a fourth aspect of students’ engagement during learning activities. Contemp Educ Psychol. 2011;36(4):257–67.

Groccia JE. What is student engagement? New Dir Teach Learn. 2018;2018(154):11–20.

Fredricks JA, McColskey W. The measurement of student engagement: A comparative analysis of various methods and student self-report Instruments. In S. L. Christenson, A. L. Reschly, & C. Wylie, editors. Handbook of research on student engagement. Springer Science + Business Media; 2012. p.763–782

Rotgans JI, Schmidt HG, Rajalingam P, Hao JWY, Canning CA, Ferenczi MA, et al. How cognitive engagement fluctuates during a team-based learning session and how it predicts academic achievement. Adv Health Sci Educ Theory Pract. 2018;23(2):339–51.

Rotgans JI, Schmidt HG. Cognitive engagement in the problem-based learning classroom. Adv Health Sci Educ Theory Pract. 2011;16(4):465–79.

Hadie SNH, Tan VPS, Omar N, Nik MohdAlwi NA, Lim HL, Ku Marsilla KI. COVID-19 Disruptions in health professional education: use of cognitive load theory on students’ comprehension, cognitive load, engagement, and motivation. Front Med (Lausanne). 2021;8:739238.

Wong V, Smith AJ, Hawkins NJ, Kumar RK, Young N, Kyaw M, et al. Adaptive tutorials versus web-based resources in radiology: a mixed methods comparison of efficacy and student engagement. Acad Radiol. 2015;22(10):1299–307.

Chan V, Larson ND, Moody DA, Moyer DG, Shah NL. Impact of 360 degrees vs 2D videos on engagement in anatomy education. Cureus. 2021;13(4): e14260.

Meng L, Jin Y. A confirmatory factor analysis of the Utrecht work engagement scale for students in a Chinese sample. Nurse Educ Today. 2017;49:129–34.

Luo Y, Geng C, Pei X, Chen X, Zou Z. The evaluation of the distance learning combining webinars and virtual simulations for senior nursing students during the COVID-19 period. Clin Simul Nurs. 2021;57:31–40.

Agarwal G, Mosquera M, Ring M, Victorson D. Work engagement in medical students: an exploratory analysis of the relationship between engagement, burnout, perceived stress, lifestyle factors, and medical student attitudes. Med Teach. 2020;42(3):299–305.

Wouters A, Croiset G, Schripsema NR, Cohen-Schotanus J, Spaai GWG, Hulsman RL, et al. A multi-site study on medical school selection, performance, motivation and engagement. Adv Health Sci Educ Theory Pract. 2017;22(2):447–62.

Casuso-Holgado MJ, Cuesta-Vargas AIM-M, N. The association between academic engagement and achievement in health sciences students. BMC Med Educ. 2013;13(33).

Liu H, Yansane AI, Zhang Y, Fu H, Hong N, Kalenderian E. Burnout and study engagement among medical students at Sun Yat-sen University, China: a cross-sectional study. Medicine (Baltimore). 2018;97(15): e0326.

Kakoschke N, Hassed C, Chambers R, Lee K. The importance of formal versus informal mindfulness practice for enhancing psychological wellbeing and study engagement in a medical student cohort with a 5-week mindfulness-based lifestyle program. PLoS ONE. 2021;16(10): e0258999.

Skodova Z, Lajciakova P, Banovcinova L. Burnout syndrome among health care students: the role of type d personality. West J Nurs Res. 2017;39(3):416–29.

Koob C, Schropfer K, Coenen M, Kus S, Schmidt N. Factors influencing study engagement during the COVID-19 pandemic: A cross-sectional study among health and social professions students. PLoS ONE. 2021;16(7): e0255191.

Puranitee P, Kaewpila W, Heeneman S, van Mook W, Busari JO. Promoting a sense of belonging, engagement, and collegiality to reduce burnout: a mixed methods study among undergraduate medical students in a non-Western, Asian context. BMC Med Educ. 2022;22(1):327.

Wu H, Li S, Zheng J, Guo J. Medical students’ motivation and academic performance: the mediating roles of self-efficacy and learning engagement. Med Educ Online. 2020;25(1):1742964.

Presoto CD, Wajngarten D, Dos Santos Domingos PA, Botta AC, Campos J, Pazos JM, et al. University engagement of dental students related to educational environment: a transnational study. PLoS ONE. 2021;16(11): e0259524.

Abreu Alves S, Sinval J, Lucas Neto L, Maroco J, Goncalves Ferreira A, Oliveira P. Burnout and dropout intention in medical students: the protective role of academic engagement. BMC Med Educ. 2022;22(1):83.

Zucoloto ML, de Oliveira V, Maroco J, Bonini Campos JAD. School engagement and burnout in a sample of Brazilian students. Curr Pharm Teach Learn. 2016;8(5):659–66.

Mennenga HA. Student engagement and examination performance in a team-based learning course. J Nurs Educ. 2013;52(8):475–9.

Cheng CY, Liou SR, Tsai HM, Chang CH. The effects of Team-Based Learning on learning behaviors in the maternal-child nursing course. Nurse Educ Today. 2014;34(1):25–30.

Ulfa Y, Igarashi Y, Takahata K, Shishido E, Horiuchi S. A comparison of team-based learning and lecture-based learning on clinical reasoning and classroom engagement: a cluster randomized controlled trial. BMC Med Educ. 2021;21(1):444.

Pickering JD, Swinnerton BJ. Exploring the dimensions of medical student engagement with technology-enhanced learning resources and assessing the impact on assessment outcomes. Anat Sci Educ. 2019;12(2):117–28.

Dixson MD. Measuring student engagement in the online course: The Online Student Engagement Scale (OSE). Online Learn. 2015;19(4):1–15.

Natarajan J, Joseph MA. Impact of emergency remote teaching on nursing students’ engagement, social presence, and satisfaction during the COVID-19 pandemic. Nurs Forum. 2022;57(1):42–8.

Hensley A, Hampton D, Wilson JL, Culp-Roche A, Wiggins AT. A multicenter study of student engagement and satisfaction in online programs. J Nurs Educ. 2021;60(5):259–64.

Chan SL, Lin CC, Chau PH, Takemura N, Fung JTC. Evaluating online learning engagement of nursing students. Nurse Educ Today. 2021;104: 104985.

Zhang S, Ma R, Wang Z, Li G, Fa T. Academic self-concept mediates the effect of online learning engagement on deep learning in online courses for Chinese nursing students: a cross-sectional study. Nurse Educ Today. 2022;117: 105481.

Clynes M, Sheridan A, Frazer K. Student engagement in higher education: A cross-sectional study of nursing students’ particpation in college-based education in the republic of Ireland. Nurse Educ Today. 2020;93: 104529.

Clynes M, Sheridan A, Frazer K. Ref: NET_2019_1563: Working while studying: The impact of term-time employment on undergraduate nursing students’ engagement in the Republic of Ireland: A cross-sectional study. Nurse Educ Today. 2020;92: 104513.

Ahlfeldt S, Mehta S, Sellnow T. Measurement and analysis of student engagement in university classes where varying levels of PBL methods of instruction are in use. Higher Educ Res Dev. 2005;24(1):5–20.

Hopper MK. Assessment and comparison of student engagement in a variety of physiology courses. Adv Physiol Educ. 2016;40(1):70–8.

Hopper MK, Brake DA. Student engagement and higher order skill proficiency: a comparison of traditional didactic and renewed integrated active learning curricula. Adv Physiol Educ. 2018;42(4):685–92.

Hopper MK, Kaiser AN. Engagement and higher order skill proficiency of students completing a medical physiology course in three diverse learning environments. Adv Physiol Educ. 2018;42(3):429–38.

Xu X, Bos NA, Wu H. The relationship between medical student engagement in the provision of the school’s education programme and learning outcomes. Med Teach. 2022;44(8):900–6.

Tcha-Tokey K, Christmann O, Loup-Escande E, Richir S. Proposition and validation of a questionnaire to measure the user experience in immersive virtual environments. Int J Virtual Real. 2016;16:33–48.

O’Brien HL, Toms EG. The development and evaluation of a survey to measure user engagement. J Am Soc Inform Sci Technol. 2010;61(1):50–69.

Salmela-Aro K, Upadaya K. The schoolwork engagement inventory: energy, dedication, and absorption (EDA). Eur J Psychol Assess. 2012;28(1):60–7.

Maroco J, Maroco AL, Campos JADB, Fredricks JA. University student’s engagement development of the University Student Engagement Inventory (USEI). Psicologia Reflexão e Crítica. 2016;29(1):158.

ASPIRE. ASPIRE student engagement guidelines for submitters and criteria2020. Available from: https://www.aspire-to-excellence.org/Areas+of+Excellence/ .

Ouimet J, Smallwood R. Assessment measures: CLASSE–the class-level survey of student engagement. Assess Upd. 2005;17(6):13.

Rossi RA, Krouse AM, Klein J. Undergraduate student stress, classroom engagement, and self-directed learning postcurricular revision. J Nurs Educ. 2021;60(10):566–9.

Szeto A, Haines J, Buchholz AC. Impact of an optional experiential learning opportunity on student engagement and performance in undergraduate nutrition courses. Can J Diet Pract Res. 2016;77(2):84–8.

O’Malley KJ, Moran BJ, Haidet P, Seidel CL, Schneider V, Morgan RO, et al. Validation of an observation instrument for measuring student engagement in health professions settings. Eval Health Prof. 2003;26(1):86–103.

Kelly PA, Haidet P, Schneider V, Searle N, Seidel CL, Richards BF. A comparison of in-class learner engagement across lecture, problem-based learning, and team learning using the STROBE classroom observation tool. Teach Learn Med. 2005;17(2):112–8.

Alimoglu MK, Sarac DB, Alparslan D, Karakas AA, Altintas L. An observation tool for instructor and student behaviors to measure in-class learner engagement: a validation study. Med Educ Online. 2014;19:24037.

Alimoglu MK, Yardim S, Uysal H. The effectiveness of TBL with real patients in neurology education in terms of knowledge retention, in-class engagement, and learner reactions. Adv Physiol Educ. 2017;41(1):38–43.

Saperstein A, Ledford CJW, Servey J, Cafferty LA, McClintick SH, Bernstein EM. Microblog use and student engagement in the large-classroom setting. Fam Med. 2015;473:204–9.

Mackintosh-Franklin C. An evaluation into the impact of undergraduate nursing students classroom attendance and engagement with online tasks on overall academic achievement. Nurse Educ Today. 2018;61:89–93.

Kauffman CA, Derazin M, Asmar A, Kibble JD. Patterns of medical student engagement in a second-year pathophysiology course: relationship to USMLE Step 1 performance. Adv Physiol Educ. 2019;43(4):512–8.

Ribeiro LMC, Mamede S, de Brito EM, Moura AS, de Faria RMD, Schmidt HG. Effects of deliberate reflection on students’ engagement in learning and learning outcomes. Med Educ. 2019;53(4):390–7.

Caton JB, Chung S, Adeniji N, Hom J, Brar K, Gallant A, et al. Student engagement in the online classroom: comparing preclinical medical student question-asking behaviors in a videoconference versus in-person learning environment. FASEB Bioadv. 2021;3(2):110–7.

Quesnelle KM, Montemayor JR. A multi-institutional study demonstrating undergraduate medical student engagement with question-type facebook posts. Med Sci Educ. 2020;30(1):111–5.

Juan S. Promoting engagement of nursing students in online learning: use of the student-generated question in a nursing leadership course. Nurse Educ Today. 2021;97: 104710.

Grant LL, Opperman MJ, Schiller B, Chastain J, Richardson JD, Eckel C, et al. Medical student engagement in a virtual learning environment positively correlates with course performance and satisfaction in psychiatry. Med Sci Educ. 2021;1133:1–8.

Berman NB, Artino AR Jr. Development and initial validation of an online engagement metric using virtual patients. BMC Med Educ. 2018;18(1):213.

Darnell DK, Krieg PA. Student engagement, assessed using heart rate, shows no reset following active learning sessions in lectures. PLoS ONE. 2019;14(12): e0225709.

Miller BW. Using reading times and eye-movements to measure cognitive engagement. Educ Psychol. 2015;50(1):31–42.

Stokes-Parish JB, Duvivier R, Jolly B. How does moulage contribute to medical students’ perceived engagement in simulation? A mixed-methods pilot study. Adv Simul (Lond). 2020;5:23.

Fredricks JA, Hofkens TL, Wang MT. Addressing the challenge of measuring student engagement. In: The Cambridge Handbook of Motivation and Learning. Cambridge: Cambridge University Press; 2019. p. 689–712.

Bicket M, Misra S, Wright SM. Medical student engagement and leadership within a new learning community. BMC Med Educ. 2010;10:20.

Reid HJ, Thomson C, McGlade KJ. Content and discontent: a qualitative exploration of obstacles to elearning engagement in medical students. BMC Med Educ. 2016;16:188.

Hartnup B, Dong L, Eisingerich AB. How an environment of stress and social risk shapes student engagement with social media as potential digital learning platforms: qualitative study. JMIR Med Educ. 2018;4(2): e10069.

Seligman L, Abdullahi A, Teherani A, Hauer KE. From Grading to assessment for learning: a qualitative study of student perceptions surrounding elimination of core clerkship grades and enhanced formative feedback. Teach Learn Med. 2021;33(3):314–25.

Fang KM, Lau GC, Park JY, Tchen P. Exploring factors that influence student engagement in community-engaged learning activities within a pharmacy context. Am J Pharm Educ. 2022;86(4):296–303.

Grijpma JW, Mak-van der Vossen M, Kusurkar RA, Meeter M, de la Croix A. Medical student engagement in small-group active learning: A stimulated recall study. Med Educ. 2022;56(4):432–43.

McNaught K, Rhoding C. Exploring the factors influencing medical student engagement with rural clinical placement opportunities. Austr Int J Rur Educ. 2022;32:70–84.

Wang Y, Ji Y. How do they learn: types and characteristics of medical and healthcare student engagement in a simulation-based learning environment. BMC Med Educ. 2021;21(1):420.

Janssen A, Shaw T, Goodyear P, Kerfoot BP, Bryce D. A little healthy competition: using mixed methods to pilot a team-based digital game for boosting medical student engagement with anatomy and histology content. BMC Med Educ. 2015;15:173.

McCoy L, Pettit RK, Lewis JH, Allgood JA, Bay C, Schwartz FN. Evaluating medical student engagement during virtual patient simulations: a sequential, mixed methods study. BMC Med Educ. 2016;16:20.

Milles LS, Hitzblech T, Drees S, Wurl W, Arends P, Peters H. Student engagement in medical education: a mixed-method study on medical students as module co-directors in curriculum development. Med Teach. 2019;41(10):1143–50.

de Leng B, Pawelka F. The use of learning dashboards to support complex in-class pedagogical scenarios in medical training: how do they influence students’ cognitive engagement? Res Pract Technol Enhanced Learn. 2020;15(1):1.

Hughes M, Salamonson Y, Metcalfe L. Student engagement using multiple-attempt “weekly participation task” quizzes with undergraduate nursing students. Nurse Educ Pract. 2020;46: 102803.

Shoepe TC, McManus JF, August SE, Mattos NL, Vollucci TC, Sparks PR. Instructor prompts and student engagement in synchronous online nutrition classes. Am J Distance Educ. 2020;34(3):194–210.

Giddens J, Hrabe D, Carlson-Sabelli L, Fogg L, North S. The impact of a virtual community on student engagement and academic performance among baccalaureate nursing students. J Prof Nurs. 2012;28(5):284–90.

Lee S, Valtis YK, Jun T, Wang D, Zhang B, Chung EH, et al. Measuring and improving student engagement in clinical training. Educ Prim Care. 2018;29(1):22–6.

Averkiou P, Nirmala P, Paiewonsky B, Sara T. Community-Engaged Learning: addressing gaps in medical education through a service-learning curriculum. J Serv-Learn Higher Educ. 2021;13:44–57.

Download references

Acknowledgements

No funding.

Author information

Authors and affiliations.

College of Medicine, Gulf Medical University, Ajman, United Arab Emirates

Salah Eldin Kassab, Mohamed Al-Eraky, Walid El-Sayed, Hossam Hamdy & Henk Schmidt

Faculty of Medicine, Suez Canal University, Ismailia, Egypt

Salah Eldin Kassab

College of Dentistry, Suez Canal University, Ismailia, Egypt

Walid El-Sayed

Institute for Medical Education Research, Erasmus University Rotterdam, Rotterdam, Netherlands

Henk Schmidt

You can also search for this author in PubMed   Google Scholar

Contributions

SEK initiated the idea of the review article. WE and SK conducted the literature search and collected the relevant articles. SEK and HH developed the draft version of the article. MA helped in editing the different versions of the manuscript. HS played a significant role in the revised version of the manuscript. All authors contributed to the revision, editing, and finalizing the manuscript. The author(s) read and approved the final manuscript.

Corresponding author

Correspondence to Salah Eldin Kassab .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Cite this article.

Kassab, S.E., Al-Eraky, M., El-Sayed, W. et al. Measurement of student engagement in health professions education: a review of literature. BMC Med Educ 23 , 354 (2023). https://doi.org/10.1186/s12909-023-04344-8

Download citation

Received : 04 January 2023

Accepted : 10 May 2023

Published : 20 May 2023

DOI : https://doi.org/10.1186/s12909-023-04344-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Student engagement
• Students as partners
• Measurement
• Health sciences
• Physiotherapy

BMC Medical Education

ISSN: 1472-6920

• Submission enquiries: [email protected]
• General enquiries: [email protected]

job description for resume customer service manager

• cover letter
• creative writing
• problem solving
• rewiew prompts

• Open access
• Published: 08 November 2023

Childhood vaccine refusal and what to do about it: a systematic review of the ethical literature

• Kerrie Wiley 1 ,
• Maria Christou-Ergos 1 ,
• Chris Degeling 2 ,
• Rosalind McDougall 3 ,
• Penelope Robinson 1 ,
• Katie Attwell 4 ,
• Catherine Helps 1 ,
• Shevaun Drislane 4 &
• Stacy M Carter 2  

BMC Medical Ethics volume  24 , Article number:  96 ( 2023 ) Cite this article

380 Accesses

9 Altmetric

Metrics details

Parental refusal of routine childhood vaccination remains an ethically contested area. This systematic review sought to explore and characterise the normative arguments made about parental refusal of routine vaccination, with the aim of providing researchers, practitioners, and policymakers with a synthesis of current normative literature.

Nine databases covering health and ethics research were searched, and 121 publications identified for the period Jan 1998 to Mar 2022. For articles, source journals were categorised according to Australian Standard Field of Research codes, and normative content was analysed using a framework analytical approach.

Most of the articles were published in biomedical journals (34%), bioethics journals (21%), and journals that carry both classifications (20%). Two central questions dominated the literature: (1) Whether vaccine refusal is justifiable (which we labelled ‘refusal arguments’); and (2) Whether strategies for dealing with those who reject vaccines are justifiable (‘response arguments’). Refusal arguments relied on principlism, religious frameworks, the rights and obligations of parents, the rights of children, the medico-legal best interests of the child standard, and the potential to cause harm to others. Response arguments were broadly divided into arguments about policy, arguments about how individual physicians should practice regarding vaccine rejectors, and both legal precedents and ethical arguments for vaccinating children against a parent’s will. Policy arguments considered the normative significance of coercion, non-medical or conscientious objections, and possible reciprocal social efforts to offset vaccine refusal. Individual physician practice arguments covered nudging and coercive practices, patient dismissal, and the ethical and professional obligations of physicians. Most of the legal precedents discussed were from the American setting, with some from the United Kingdom.

Conclusions

This review provides a comprehensive picture of the scope and substance of normative arguments about vaccine refusal and responses to vaccine refusal. It can serve as a platform for future research to extend the current normative literature, better understand the role of cultural context in normative judgements about vaccination, and more comprehensively translate the nuance of ethical arguments into practice and policy.

Peer Review reports

Introduction

Vaccine rejection has existed for as long as vaccines [ 1 ]. Despite the significant contribution of childhood vaccination to reductions in global child morbidity and mortality [ 2 ], some parents continue to reject vaccines for their children. Parents’ reasons for rejection vary widely, and often depend on their social settings. For example, in high-income settings where around 2–3% of parents reject routine childhood vaccines [ 3 , 4 ], reasons can include previous bad experiences with vaccines or the medical system, concerns about vaccine safety, doubt about the effectiveness or necessity of vaccines, alternative health approaches, and participation in particular social groups or communities. These reasons can be grounded in deeply held religious beliefs or general philosophical approaches to health, views on freedom of choice, or mistrust in government and/or the vested interests of vaccine producers, among other things [ 5 , 6 , 7 , 8 ].

Vaccination plays a dual role in disease prevention: it serves to protect the vaccinated individual from disease, and when vaccination rates reach a high enough threshold for some diseases, also protects the broader community—including those who remain unvaccinated—by disrupting disease transmission through herd immunity. This dual role of vaccination, providing benefit to both the individual and community, complicates ethical questions regarding vaccine refusal, specifically, whether vaccine rejection is ethically justifiable.

Health care providers, communities, and governments encourage uptake and discourage vaccine rejection by various means, and the dual role of vaccination is also relevant to an evaluation of these practice and policy responses. Vaccine acceptance is encouraged with interventions like incentives, health provider recommendations and “nudges” directed at individual families, as well as by facilitating easier access to vaccination through strategies such as cost reduction and making clinic locations and opening times convenient, with many of these interventions supported by varying levels of evidence [ 9 ]. Governments often discourage vaccine rejection via the imposition of mandates that can vary in type and severity [ 10 ] and are not always well-supported by evidence [ 11 ]. These can include punitive measures, such as limiting unvaccinated children’s access to early childhood education or daycare. A thorough understanding of the ethical dimensions of childhood vaccine rejection and responses to it is important when navigating vaccine rejection in the clinical setting, and when formulating policy [ 12 ]. Systematic reviews of the evidence are considered best practice for informing vaccine practice and policy however, to our knowledge there have not yet been any published systematic reviews of the literature on the ethics of childhood vaccine rejection despite there being a broad literature on the subject. We sought to systematically explore and characterise the normative arguments made about parental refusal of routine vaccination, with the aim of better informing vaccine policy and practice.

We searched nine databases for literature that discussed normative positions on childhood vaccine rejection. Refer to the PRISMA flow chart (Fig.  1 .)

PRIMSA Flow Diagram of Review

Search strategy

We searched Medline, Embase, Philosophers Index, Philpapers, Project Muse, Cinahl, The Global Digital Library on Ethics (globethics.net), The Bioethics Literature Database (BELIT), and Pubmed using the general search strategy listed in Fig.  2 for articles published between January 1998 and March 2022.

Inclusion criteria

We included any publication which provided a substantive normative argument about parental refusal of routine vaccines for children aged five and under. We used a broad definition of ‘normative’ to mark anything that goes beyond mere description to consider right and wrong, good and bad, justifiable and unjustifiable, or legitimate and illegitimate actions or ways of being in the world. Our broad conception included textual forms such as ethical reflections, prudential and legal norms, and accounts of rationality. We used ’substantive’ to mark publications where the authors’ main purpose was to make an argument about whether vaccine refusal is morally justifiable. This included empirical research that explicitly examined normative dimensions of vaccine refusal. We were limited to reviewing publications published in English.

Exclusion criteria

We excluded publications where authors made a normative claim in passing, but the publication’s main purpose was to report non-normative empirical findings. We also excluded: publications on adult vaccination (including COVID vaccination) and the HPV vaccine (which is administered in adolescence, not childhood); empirical research such as surveys or interviews, unless they expressly explored normative arguments; and descriptive publications about the characteristics of the anti-vaccination movement that provided no normative position.

Screening and data extraction

After search execution and duplicate removal, a screening triangulation exercise was undertaken to ensure consistency among the screeners. A set of 20 titles and abstracts were independently screened by six authors, and the results compared. The inclusion and exclusion criteria were refined in a subsequent group discussion, and a sub-set of full text articles were then screened and evaluated by the same group of people, and results again compared. A discussion of this second triangulation step resulted in a refined and standardized screening approach.

The authorship group were then divided into four pairs, and the remaining titles and abstracts divided among the pairs. Each individual screened titles and abstracts against inclusion criteria, and then met with their screening partner to compare results and discuss and resolve any differences.

Full text was sought for each record screened for inclusion, and a second screening then removed articles which didn’t meet the inclusion criteria once the full text was read, articles that could not be sourced, and duplicates not identified in the initial screening.

The final list of full text publications was then divided among four authors (SC, RM, CD and KW) for data extraction using the concept of “information units” described by Mertz and colleagues [ 13 ]. In this context an information unit was defined as a normative issue or argument, and each of the four ‘extracting’ authors summarized each of the relevant information units in the papers they were assigned.

For included journal articles, Australian Standard Field of Research (FoR) codes for the journal that each article appeared in were sourced as a proxy for the disciplinary location of the article (e.g. bioethics, medicine, law). We used the Australian and New Zealand Standard Research Classification (ANZSRC) 2008, as this was the current standard when analysis commenced [ 14 ]. We used two digit FoR codes (division codes) to identify the source journal as either being Medical and Health Sciences (code 11), Ethics and Philosophy (code 22) Law (code 18) or other codes grouped as “other”. In some cases, the journal was assigned a combination of these codes (refer to Fig.  3 ).

Respective percentages of included articles falling under various ANZSRC FoR Codes (2008)

Quality assessment in systematic reviews of normative literature remains a contested area, with various options and no established best practice approach [ 15 ]. In this review, we took a satisficing approach to quality appraisal [ 16 ]: publication in a peer-reviewed journal or by a reputable academic publisher was taken as a sufficient level of quality to justify inclusion in the review. The peer review process undergone by PhD theses was also taken to be a sufficient indictor of quality to justify inclusion. Further quality appraisal of individual publications was not undertaken. This aligns with the purpose of the review which was to map and synthesize the current literature on this topic.

A framework approach was used to organise and synthesise the data [ 17 ]. The extracted information units were read by one author (KW), and a coding frame inductively developed to summarise and classify the information units extracted by the group. The publications were then independently coded according to this framework by two authors (KW and PR). Following this, the two authors met and compared their coding, discussing any differences and resolving them by consensus. The data were then synthesized into themes. In addition, for journal articles, the ANZSRC Field of Research codes for the journal each article appeared in were descriptively analysed to assess the distribution of the included literature across various disciplines.

Search results

Five thousand, two hundred and thirty-one publications were returned by the searches (see Fig.  1 ). Eight hundred and twenty-two duplicates were removed in the first instance, leaving 4409 records to be screened by title and abstract. During this screening process 4058 records were excluded, leaving 351 full text publications to be assessed. Of these a further 230 records were excluded (due to not meeting the inclusion criteria, previously unidentified duplicates, or inability to source the full text), leaving 121 publications for inclusion in the review. These included 117 journal articles, three theses and one book.

Literature source type

Analysis of the ANZSRC Field of Research codes of the source journals of included articles revealed three main areas, or a combination of them (Fig.  3 ). Around half were coded to medicine (63%); of these, just over half were dual coded to ethics (20%) or another code (9%). 21% of articles were from the philosophy or ethics literature alone; another 25% were from ethics and medicine or ethics and law. Law was the least dominant discipline, with only 12% of articles being coded to law (alone or in combination with other disciplines). This pattern suggests active concern within medicine regarding non-vaccination, but also widespread overlap in concern between medicine, ethics, and law.

Main themes found in the literature

Articles addressed two central questions (see Table  1 ):

Whether vaccine refusal was justified (henceforth ‘refusal’ arguments).

Whether various policy or practice responses to those who reject vaccines are justified (henceforth ‘response’ arguments).

Descriptive analysis of content

The literature was dominated by papers focused on ‘response’ arguments (61%). A smaller group of papers address ‘refusal’ arguments (19%), and about 18% considered both ‘refusal’ and ‘response’, usually making normative arguments about vaccine refusal as background to arguments regarding ‘response’ (See Fig.  4 ). Less than 2% of papers had a different focus.

Comparative frequencies of themes occurring among included articles

‘Response’ arguments were more common in the medical and health sciences literature (ERA FoR code 11, see Fig.  5 ). Although the ethics/philosophy (FoR code 22) and law literatures (FoR code 18) were also dominated by ‘response’ arguments, these journals—unlike medical journals—were more likely to include ‘refusal’ arguments.

Comparative frequency of overarching themes across the different disciplines of the included articles

As would be expected, authors made ‘response’ and ‘refusal’ arguments in different ways. In the following sections we consider the detail of how arguments were made. We refer to each included article by its unique reference listed in Table  1 .

‘Refusal’ arguments: whether or not vaccine rejection by individual parents is justifiable

Arguments about whether vaccine refusal by individual parents is justifiable included consideration of parents’ rights, the interests of the child (including the legal ‘best interests of the child standard’), the value of herd immunity, the epistemic basis for ethical claims, and the relevance of religious views. Our sampling period included a special issue of Narrative Inquiry in Bioethics which published narratives written by parents to communicate their normative positions on vaccination. Most of these were written by non-vaccinating parents, and they make up over one third of all arguments in the identified literature that support refusal. On balance, most of the literature argues that it is not justifiable for parents to refuse routine vaccination for their children.

Some arguments within the literature were absolute in their position on whether vaccine rejection is justifiable; others weighed competing values in a situation-specific approach. Irrespective of the arguments used to justify a position, most of the literature frames the question of whether vaccine rejection is justifiable based on three key areas of concern: (i) Respect for autonomy, the doctrine of informed consent and the value of liberty, (ii) Consequences for the child and others, and/or (iii) The normative significance of parental trust, distrust, and uncertainty. We explore the main arguments within these concepts below. As the discussion shows, these concepts are not discrete – they are often weighed against one another, linked by causal claims, or held in tension in the arguments made. Figure  6 represents proportionally the ’refusal’ arguments made in the reviewed literature.

‘Refusal’ arguments made in the literature on the ethics of vaccine refusal

Respect for autonomy, the doctrine of informed consent and the value of liberty

Fifteen papers from this sample present arguments that vaccine refusal is justified based on respect for parental autonomy, rights, or liberties (21, 23, 25, 31, 32, 35, 36, 39, 68, 71, 75, 80, 94, 100, 121). Some argue that vaccine refusal is justified on the basis of preserving legal rights (31, 80) or expression of religious freedom [ 23 ]. Opposing positions (including from four of the authors who also offer arguments justifying refusal) argue that, on balance, considerations regarding respect for autonomy are, or can be, outweighed by the potential harm caused to the child and others by not vaccinating though the increased risk of vaccine preventable diseases (21, 36, 20, 23, 110). This includes legal perspectives arguing that the freedom to choose is not unfettered [ 25 ] and that courts can override parental autonomy if this is in the child’s best interest (75, 85), as well as arguments from religious perspectives that the freedom to exercise religious beliefs needs to be weighed against harm caused to others (21,91). Those who argue that vaccine refusal is justified counter that disrespecting parental autonomy can also cause harm to the child through loss of trust and possible disengagement of the child from the healthcare system (100), and that the increased risk of disease is a price worth paying to ensure that political values are preserved (71). Of note: non-vaccinating parents also assert a right to make choices for their children in support of their refusal [ 14 , 18 ], but unlike others, their arguments are based primarily on epistemic claims about vaccine effectiveness, necessity and safety rather than moral or ethical positions. However, they assert that these doubts necessitate respect for their decision.

Consequences for others and the child

Most of the literature argues for or against the justifiability of vaccine refusal based on consequences. These include potential harms from vaccine preventable diseases or vaccines themselves, or conversely, potential benefits from herd immunity. The concept of herd immunity is deployed in different ways. Those justifying vaccine refusal in certain circumstances argue that in settings where there is a high level of herd immunity, the risk posed by an unvaccinated child is not great enough to override respect for parental autonomy (62, 65, 94, 98), and that the benefits of community protection do not justify the individual risk posed by the vaccine and borne by the child who is already protected through herd immunity (72, 96, 97, 17, 93, 108). Perspectives of non-vaccinators echo these ideas by asserting that some diseases are not harmful enough to proscribe vaccine refusal [ 14 ] and that vaccine injury contributes to and justifies refusal [ 16 ].

In contrast, those who argue that refusal is not justifiable propose a duty to contribute to herd immunity because it is a public good (7,80, 19,120, 33, 48, 68,115), or that free-riding (allowing one’s child to enjoy the benefits of herd immunity provided by others, while avoiding the risk of vaccinating) is unfair (37,46, 48). On this account, the vaccine refusal of a few may undermine herd immunity and thus cause harm to the many by increasing disease risks (9, 11, 26, 37, 59, 76, 81, 86); further, these risks are borne by the most vulnerable (43). These arguments about harm to others include those made by authors writing from religious perspectives (8, 81, 84, 92, 98). Finally, an account by a vaccinating parent suggests that harms resulting from non-vaccination are blameworthy because they are an intentional act of aggression against vaccinated children [ 19 ].

The concept of the child’s interests arises frequently in these publications. Pursuing or protecting these interests generally combines concern about the consequences of non-vaccination for the child with concern for autonomy, in the broad sense of being able to direct one’s life in accordance with one’s values or aims. Authors write about the interests of the child in both a general sense (i.e. the interests of the child outside of a legal context) and in a legal sense (the formal ‘best interests of the child standard’). The legal construction is used both to support (31, 6, 93) and to oppose vaccine refusal. Arguments that receiving a vaccine is in the legal ‘best interests of the child’ (21,39) posit that any deviation from a widely accepted legal view of the interests of a child should weigh the risk of harm to the child (68) irrespective of the parent’s beliefs (78), or that non-vaccination constitutes negligence or child endangerment [ 28 ]. On the other hand, some authors argue that, from a legal perspective, parents have the right to consent to or refuse vaccination ostensibly using the ‘child’s best interests standard’(93) and that there is insufficient legal precedent to argue that non-vaccination constitutes medical neglect [ 6 ].

Arguing from distrust and uncertainty

As previously noted, the sample included a set of papers written from the perspective of non-vaccinating parents. Most of these contributions seek to justify vaccine refusal, and many justifications were grounded in distrust. They call into question vaccine safety and effectiveness [ 12 , 13 , 14 , 18 ], and the accuracy of the reporting of adverse events following immunization (96). They claim financial conflicts, constructing clinicians, clinical medicine, and/or regulatory agencies as untrustworthy or non-credible [ 12 , 14 , 16 ]. They cite empirical studies of non-vaccinators to support parental preferences for natural infection over a vaccine (97). Non-vaccinating parents were not the only authors to make arguments in this vein. Some other authors cite the lack of absolute certainty of vaccine safety as justification for parents refusing vaccines in the interests of their children (28,76), especially regarding newer vaccines for which efficacy is not well-established (34). This line of argument depicts vaccine proponents as driven by commercial interests, thus justifying parental mistrust and refusal (34). Contra this, one paper asserts that refusal on the grounds of mistrust of government or medicine is not justifiable, as it is inconsistent with the scientific evidence and the well-established regulatory processes in place, such as the rigorous clinical testing required to develop and approve vaccines, and the systems established to report adverse events and ensure safety [ 8 ].

‘Response’ arguments: claims regarding the justifiability of different responses to non-vaccination

The literature examines four main responses to non-vaccination (i) government mandate policies (such as legal ramifications for refusing vaccination and vaccination as a school entry requirement), and other coercive policies, (ii) exemptions to mandate policies, (iii) individual practitioner and medical practice responses (including patient dismissal from practice for vaccine refusal, vaccinating against parents’ will, and nudging), and (iv) withholding health resources. The literature includes authors who argue that these responses are justifiable and others who argue that they are not. Much like the refusal arguments, some response arguments are absolute in their position, while others advocate weighing competing values in a context -specific way. Like refusal arguments, most arguments for and against particular responses to non-vaccinating parents draw from respect for autonomy, the doctrine of informed consent and the value of liberty, as well as considering consequences for the child and others. Other concepts appearing in these arguments include inequity, and the duties of governments and practitioners. Figure  7 represents proportionally the ’response’ arguments made in the reviewed literature.

‘Response’ arguments made in the literature on the ethics of vaccine refusal

As in the literature on refusal, many arguments about policy or practice responses to non-vaccinating parents depend on the interrelated concepts of respect for autonomy, informed consent and liberty. Five papers engage with the issue of practitioners vaccinating against parents’ will with respect to these concepts. They argue that forced vaccination by healthcare providers violates parents’ autonomy and/or the ethical requirement for informed consent, because vaccination carries risks (80,119), and clinicians have legal obligations to obtain valid consent for procedures (94). Some authors propose alternatives to forced vaccination, including focusing on rebuilding trust (rather than violating negative liberty) (32), and accepting that views on vaccination derive from plural and culturally-specific values [ 29 ]. On the other hand, proponents of forced vaccination do not engage with these concepts, instead deploying the harm principle and the legal ‘best interests of the child standard’ to justify their position. We explore this argument in the following section “Consequences for the child and others”.

Another set of papers make arguments about vaccine mandates that also draw on autonomy or liberty justifications, often weighing these against harm or risk of harm. Arguments justifying mandates are often legal in nature and use, for example, the harm principle or case law to argue that the freedom or liberty to choose not to vaccinate is limited by the risk of ill health and/or death to the child or others in the community, including vulnerable persons (83,91). One author argues that legal actions should be brought against those who harm others by refusing vaccination, as this would both discourage refusal and, in the case of any successful claims, compensate victims (55). Some authors argue that mandates are justifiable if the exercise of liberty rights poses a threat to public health (53,82,83,91,119). While those arguing that mandates are not justifiable sometimes rely on arguments about risk of harm—i.e. that in a low-incidence (and therefore low-risk) setting mandates cannot be justified (45, 87,104)—most make their arguments from autonomy, informed consent, and personal liberty and do not weigh these against the potential for harm (12,16,61,82,89,107,114). One author argues that even if mandates improve vaccination rates, they damage trust with parents and make refusers more steadfast in their decision (121), so are not sustainable. Finally, some authors present middle-ground positions that—in their view—are more autonomy- or liberty-preserving, including persuasion (121) or weakly enforced mandates (71), or argue that policy responses should take the least coercive approach that is feasible and effective to balance the needs of the individual with public health (117).

Those supporting conscientious objection to mandates argue that such provisions contribute to the collective good of a culture of respect for autonomy (82), or reflect the “American ideal” of personal freedom (66). Contra this, those opposed to conscientious objection provisions argue that challenges to mandates based in religious freedom have failed in case law, as the right to practice religion freely does not include the liberty to expose children or communities to disease (20,92). One author provides a qualified view of conscientious objection on religious grounds, arguing that such liberties could be justified only while high vaccination rates are maintained (109).

Authors disagree about whether certain policy or practice responses do, or do not, respect autonomy or uphold important liberties. For example, authors disagree on the effect of both nudges and conscientious objection policies on parental autonomy or liberty. With respect to nudges, some argue they are autonomy-preserving because they steer parents in a certain direction while allowing choice (106), do not override or challenge the strong views of deeply opposed opponents (42, 44) and uphold informed consent (121). Some supporters of nudging weigh multiple normative considerations, arguing that nudges that appeal to social responsibilities in a medical practice setting are justified because they appropriately balance parental autonomy against the practitioner’s responsibility to promote trust and collective benefits (3,80). Those opposed to nudges for vaccination decisions argue that the invasive nature of immunization increases the need for independent and informed decision making (60,113). These authors argue against a presumptive consultation style in general practice, proposing participatory clinical encounters (114), and using persuasion (42), as alternatives to more coercive approaches.

Consequences for the child and others

Many of the arguments in this literature consider individual and collective consequences—benefits, harms, burdens, and costs to society — and propose that these may override other normative considerations. The risk and prevention of harm is particularly pertinent here. For example, a parental decision can be overruled in cases where there is a significant risk of harm to the child (78), or nudges become more justifiable when the risk of harm to others is higher (3, 75).

Arguments about mandates often include concern about consequences, since it is inherent in a vaccine mandate that there will be some costs associated with non-vaccination. Mandate proponents argue that mandates ensure high vaccination rates, thus preventing disease outbreaks (39) and associated harms (97), so are in the best interest of individual children (28, 73, 111) and serve the greater good (4,28,73,79). Some justify mandates by proposing a duty to contribute to herd immunity, including under the “clean hands principle”, that is, an obligation not to participate in collectively harmful activities [ 1 , 5 ]. Conversely, some authors argue that mandates are not necessary to achieve high levels of population immunity, so state coercion is unjustified at a collective level or at the level of the individual child because each child receives limited benefit (94). Those opposing mandates also argue that vaccine safety is not absolute (88) and that mandates are a disutility, carrying associated costs with surveillance and enforcement (95). Other authors sought to balance these kinds of consequences against other normative considerations with respect to mandates, including the level of herd immunity, the risks of non-vaccination to the child and/or society, and respect for parental autonomy (32,53,88,119). One author argues that mandates protect ‘victims’ of the anti-vaccination movement from harms so long as certain conditions are met (43): that the vaccine can prevent infection and transmission, that individuals minimize their risk of exposure, and that the right of self-defense is preserved (e.g. in the case of allergy to vaccines).

Consequences are also important to arguments about conscientious objection, but here it is generally concerns about the impact on the collective. Some argue that exemptions should not be allowed because they may increase rates of disease or undermine individual or community health (20, 87, 118); others argue that if disease risk is low, exemptions are justified because those few individuals with exemptions do not pose a risk to others or herd immunity (20, 82, 105).

Consequences to the child and others are used to justify whether responses should be applied in general practice settings. As mentioned in the previous section, some authors justify healthcare workers vaccinating against a parent’s will using both the harm principle (69) and the legal ‘best interests of the child standard’ [ 25 ]; others suggest it is against the legal best interests of an older child to be forcibly vaccinated, as this may have a more detrimental impact than being unvaccinated (25,51). The best interests of the child are also invoked extensively to argue that non-vaccinating families should not be dismissed from medical practices (98,104, 26, 75). Here authors note that an unvaccinated child is more vulnerable to vaccine preventable diseases (9, 49), practice dismissal limits opportunities to access health care (31,52, 56,79,116) and the increased risk of harm from vaccine preventable diseases is transferred to other practices (9,47,49). One paper makes an argument about the consequences of treating non-vaccinating families for general practitioners, suggesting that practices caring for unvaccinated children should disclose this to other patients to minimize medicolegal risks, and should receive legal protection to account for the increased liability and risk of caring for these patients (40).

A small body of literature employs claims about who is responsible for the consequences of non-vaccination to make arguments about responses to non-vaccination. For example, one article seeks to justify discriminating against unvaccinated children with a vaccine preventable disease by limiting their access to health resources, relying on precedents such as coronary bypass surgery being withheld from obese people and smokers, and arguing that those who contribute to their own ill-health (in this case by not vaccinating) do not deserve healthcare (80). A related argument focuses on managing refugee camps during outbreaks that pose a direct and imminent threat of harm, proposing that the state is justified in withholding humanitarian aid from non-vaccinating refugees because the state is responsible for setting conditions that provide protection to (or prevent harm to) aid givers and public health [ 30 ].

Some critiques of policy or practice responses to non-vaccination emphasise that these responses can have inequitable effects and argue that this is unjustifiable. Exemption policies are a key focus here. Five papers argue against exemptions to vaccine mandates on the grounds that these unevenly distribute the risks and benefits of vaccinations (27,61,66, 73,118). These authors propose that the inaction of a few compromises the health of the most vulnerable community members (118) and disenfranchises those with medical contraindications for vaccines [ 27 ]. One author particularly focuses on home-schooled children, arguing that exempting them from vaccine mandates exposes both those children and society to harm, and that it is in the interests of these children and society that they be protected through vaccination (73). Some authors suggest that policy exemptions could be made justifiable by imposing conditions that offset potential inequities. On this view, exemptions could be justified so long as the refuser is prepared to make a financial or other contribution to help offset the potential financial burden of the diseases they may cause, or to otherwise contribute to social good [ 2 , 22 ].

Similarly, some opponents of coercive mandates or practice dismissal for non-vaccination critique these responses for having inequitable effects. It is argued that coercion risks creating a group of disenfranchised people (113) and that different people have different capacities to resist coercive policies (114). Similarly, dismissal leaves vulnerable children without advocacy (64), leads to patients not being treated equally (63) and marginalizes children from health care (74). One paper argues that family dismissal should be strongly discouraged, and an alternative mutually beneficial solution sought after considering the interests of the patient, physician, family, community, and society at large (74).

The duty of practitioners and the state

Some papers address the duties of practitioners and the duties of the state to respond to non-vaccination, in ways that go beyond simply weighing up consequences, implications for autonomy or freedom, or equity of impacts.

A variety of duties of practitioners are proposed. The first of these is to protect a child from their parent’s beliefs if those beliefs are likely to cause significant harm, which is used to justify initiating child protection proceedings to vaccinate against a parent’s will (67). Another is to protect patients in the waiting room from the risks posed by non-vaccinating patients, which is used to justify dismissing non-vaccinating patients from practice (9,26,38, 40,45). Counter-obligations are used to argue against practice dismissal. These include a health professional’s obligation to provide healthcare in the best interest of the child despite the parent’s decisions, and to deal with infectious disease as a part of their role (9,26,45,47, 56,101). Authors also argue that physicians’ obligations exclude enforcing parental accountability through dismissal, especially if that means the child is held accountable for the actions of their parents (47), and that continuing to provide care to a non-vaccinating family does not make the physician complicit in their decision (116).

It is sometimes asserted that the state is obliged to discourage non-vaccination on a number of grounds. This includes a fundamental duty of states to protect society [ 21 ], a responsibility of states to protect herd immunity as a common good or to reduce social and financial burdens and costs (53,119), and the state’s role to protect the common good in the face of risks to public health and the fallibility of individuals’ risk perception (54). Some of these arguments focus on exemptions from mandatory vaccination policies, proposing that states can not justify such exemptions because the government’s interest in protecting society outweighs the individual’s interest [ 21 ] or because vaccination is a social and moral good owed by a society to its children (118).

This review systematically explored and characterised the normative arguments made about parental refusal of routine childhood vaccination. Included publications addressed two types of arguments (i) ‘Refusal’ arguments (whether vaccine refusal is justified) and (ii) ‘Response’ arguments (whether various policy or practice responses to those who reject vaccines are justified). There were more ‘response’ arguments than ‘refusal’ arguments in the literature. On balance, most of the literature on ‘refusal’ arguments contended that it is not justifiable for parents to refuse vaccination for their children. Most of the ‘response’ argument literature argued against the various responses to non-vaccination put forward. However, compared to ‘refusal’ arguments, ‘response’ arguments were more varied and nuanced, and often came with caveats (e.g. exemptions to mandates are permissible if the disease burden is low).

The included articles predominantly originated from medical journals: these accounted for most of the papers focused on ‘response’ arguments. This may arise from the broader distribution of academic literature – there are more papers published in medicine than in the other disciplines represented in this review. It may also reflect the needs of readers of medical literature for guidance on how they should respond to non-vaccinating parents, highlighting the importance of making literature addressing the ethical dimensions of vaccine refusal accessible to immunization practitioners. Although there were some interdisciplinary perspectives, the dominance of the medical literature relating to ‘response’ arguments suggests that knowledge in this field may be advanced by incorporating more voices with expertise in ethics, law, and policy. This is especially important for deciding how to implement policy and practice responses to non-vaccination.

‘Refusal’ arguments were more common in the comparatively smaller collection of ethics/philosophy literature identified by this search, which may be, in part, a product of the differences in disciplinary traditions. While ethics/philosophy texts explore counterarguments and reach conclusions that are nuanced, and often with caveats, medical disciplines are primarily guided by practical considerations and a tradition of arguing from evidence rather than from ethical or philosophical principles. This privileging of evidence over principles may make it difficult to explore differing vaccination positions within the medical arena, potentially contributing to the adversarial clinical immunisation encounters described by vaccine-refusing parents and clinicians alike [ 7 , 18 , 19 ]. This pattern needs attention if ethical arguments are to have an impact in practice. As shown, most ethical arguments pay attention to evidence, as most ethical arguments include consequences in some way (see below). Ethical arguments can add nuance to biomedical thinking about consequences (e.g. consequences for individuals vs. the collective) and also about competing values (e.g. balancing consequences against concerns regarding autonomy, consent and liberty). The challenge for ethicists is to provide these arguments in an accessible and compelling form.

In fact, (i) consequences for the child and others, and (ii) respect for autonomy, the doctrine of informed consent and the value of liberty were dominant themes in both ‘refusal’ and ‘response’ arguments. Arguments were guided by common concepts such as the value of herd immunity, the prospect of harm to the child or others in the community and legal perspectives and precedents. The normative significance of parental trust, distrust, and uncertainty was a consideration unique to the ‘refusal’ arguments literature, driven in part by the five parental accounts from the special issue of Narrative Inquiry in Bioethics included in our sample. The concepts of inequity, and the duties of governments and practitioners only appeared in ‘response’ arguments. This is unsurprising: it reflects the purpose and perspective of these writers. An analysis of policy options is often required to bring inequity into view, and both clinicians and policymakers have obligations by virtue of their roles that can inform thinking about the right thing to do.

Many of the arguments justifying vaccine refusal aligned with the wider literature on the perspectives of non-vaccinating parents who value the freedom to make health decisions as caregivers, in what they perceive to be the best interest of their children [ 20 , 21 ]. These decisions are often based on doubts about vaccine safety or efficacy and are commonly initiated by a negative experience [ 19 , 20 , 22 ]. Unsurprisingly, arguments against rejecting childhood vaccines reflected the broader literature on how vaccine-supporting people view non-vaccination— including views that non-vaccinators are misinformed and disrupt social order, and that their actions are not based on reason or shared social values [ 23 ]. Common negative descriptors such as “anti-vaxxer” have similar valence in social discourse [ 24 ]. Those writing about vaccination should be aware of the potential for stigmatization and “othering” that can result by framing non-vaccination as a failure of parents [ 25 ]. When such arguments are used to inform policy and practice responses to non-vaccination, it introduces the potential for negative psychosocial impacts and further alienation of non-vaccinating parents.

Most ‘response’ arguments dealt with the justifiability of mandates and coercive policy. Generally, authors in favour of mandates prioritised the good of society; those against mandates prioritised individual choice. The large number of papers we found on mandates is unsurprising, given that these policies have been contentious. In Australia, federal and most state governments have mandates that require children to be vaccinated to be enrolled in childcare and for their families to be eligible for government financial assistance [ 26 ] Key political, academic and industry stakeholders argue that these mandates are designed to increase vaccination rates for the benefit of society [ 27 ]. On the other hand, Australian non-vaccinating parents express a belief that their children do not pose a threat to society, that all children should be treated in the same way, and that all parents should be able to make decisions for their children, regardless of vaccination status [ 28 ]. These perceptions of policy makers and non-vaccinating parents broadly represent the opposing arguments about mandates presented in this review. Facilitating a middle-ground approach to policy implementation may require closer attention to the values underlying these opposing views, and using a procedurally just approach to weigh them against one another.

In the context of an increasing number of systematic reviews in the field of bioethics, there has been recent criticism emerging about the use of these methods in bioethics. For example, Birchley and Ives (2022) argue that such methods are designed and therefore better suited to aggregation of quantitative data and not the complex and subjective nature of bioethical concepts and the theory-generating and interpretive approaches they require [ 29 ]. We argue that our application of the framework systematic review method - one of many well-established methods for systematic review and synthesis of qualitative and conceptual data - is appropriate for this research question and the application of our findings. Vaccine policy and practice requires a synthesis of what is known on relevant issues, and a systematic approach such as that used here provides a useful summary of the breadth of relevant ethical issues in a format that is accessible to policymakers. Our review has some limitations. Our aim was to map the range of normative arguments about vaccination refusal and policy. We did not have scope to present a novel ethical argument in response to our findings; this is an aim for future empirical and theoretical research. Most of the included literature focuses on high-income settings, predominantly the United States and the United Kingdom. In low-income settings, health services are often harder to access and levels of and reasons for vaccine rejection also differ in these settings. For example, political and cultural factors have been implicated in polio vaccine rejection in Nigeria [ 30 ], while low literacy, unemployment, and owning a mobile phone have been associated with polio vaccine refusal in Pakistan [ 31 ]. Our sampling period included a special issue of Narrative Enquiry in Bioethics which published narratives written by parents to communicate their normative positions on vaccination. These were mostly written by non-vaccinating parents and made up over one third of all arguments in the literature that support refusal. This is a strength in that it expanded the range of views represented in the review. However, it is also a limitation in that if this special issue had not been published within our sampling period, the range of arguments would have been more strongly skewed against vaccine refusal. These papers artificially increased the proportion of arguments in the scholarly domain that argue for vaccine refusal. It is a strength of our methodology that we were able to identify the unique perspective from which they were written and position them separately in our literature synthesis so that our representation of the literature distribution is not artificially skewed.

This review highlights an opportunity for interdisciplinary collaboration to widen the scope and reach of normative arguments about non-vaccination. Such collaboration can facilitate a broader understanding of and engagement with the ethical issues that may be relevant for practitioners, policymakers, and researchers in deciding how to respond to non-vaccinating parents. Arguments about the justifiability of non-vaccination and what should be done about it have the potential to positively influence routine childhood vaccination rates but can also alienate non-vaccinating families if not deployed with their perspectives in mind. There is an avenue for future work to further understand the influence of cultural context on normative arguments, especially within low- and middle-income settings. Moreover, there is an opportunity to further explore the influence and translation of scholarly ethical arguments into policy and practice responses to childhood non-vaccination.

Data Availability

The datasets generated and/or analysed during the current review are not publicly available, however the search terms used to generate the dataset are included in this published article.

Spier RE. Perception of risk of vaccine adverse events: a historical perspective. Vaccine. 2001;20:78–S84. https://doi.org/10.1016/S0264-410X(01)00306-1

Article   Google Scholar  

Nandi A, Shet A. Why vaccines matter: understanding the broader health, economic, and child development benefits of routine vaccination. Hum Vaccines Immunotherapeutics. 2020;16(8):1900–04. https://doi.org/10.1080/21645515.2019.1708669

Siddiqui M, Salmon DA, Omer SB. Epidemiology of vaccine hesitancy in the United States. Hum Vaccines Immunotherapeutics. 2013;9(12):2643–48. https://doi.org/10.4161/hv.27243

Beard FH, Hull BP, Leask J, et al. Trends and patterns in vaccination objection, Australia, 2002–2013. Med J Aust. 2016;204(7):275–75.

Ward PR, Attwell K, Meyer SB, et al. Understanding the perceived logic of care by vaccine-hesitant and vaccine-refusing parents: a qualitative study in Australia. PLoS ONE. 2017;12(10):e0185955.

Ward PR, Attwell K, Meyer SB, et al. Risk, responsibility and negative responses: a qualitative study of parental trust in childhood vaccinations. J Risk Res. 2017;1–14. https://doi.org/10.1080/13669877.2017.1391318

Wiley KE, Leask J, Attwell K, et al. Parenting and the vaccine refusal process: a new explanation of the relationship between lifestyle and vaccination trajectories. Soc Sci Med. 2020;263:113259.

Yaqub O, Castle-Clarke S, Sevdalis N, et al. Attitudes to vaccination: a critical review. Soc Sci Med. 2014;112:1–11.

Brewer NT, Chapman GB, Rothman AJ, et al. Increasing vaccination: putting psychological science into action. Psychol Sci Public Interest. 2017;18(3):149–207.

Attwell K, Navin MC, Lopalco PL, et al. Recent vaccine mandates in the United States, Europe and Australia: a comparative study. Vaccine. 2018;36(48):7377–84.

Attwell K, Seth R, Beard F et al. Financial interventions to increase vaccine coverage. Pediatrics 2020;146(6).

Attwell K, Navin M. How policymakers employ ethical frames to design and introduce new policies: the case of childhood vaccine mandates in Australia. Policy & Politics. 2022;50(4):526–47. https://doi.org/10.1332/030557321x16476002878591

Mertz M, Strech D, Kahrass H. What methods do reviews of normative ethics literature use for search, selection, analysis, and synthesis? In-depth results from a systematic review of reviews. Syst Rev. 2017;6(1):261. https://doi.org/10.1186/s13643-017-0661-x . [published Online First: 20171219].

Australian Bureau of Statistics., 2008, Australian and New Zealand Standard Research Classification (ANZSRC), availabe at: https://www.abs.gov.au/ausstats/[email protected]/0/4ae1b46ae2048a28ca25741800044242?opendocument

Kahrass H, Borry P, Gastmans C et al. PRISMA-Ethics – Reporting Guideline for Systematic Reviews on Ethics Literature: development, explanations and examples 2021 [Available from: OSF Preprints at https://osf.io/g5kfb

McDougall RJ, Notini L. Overriding parents’ medical decisions for their children: a systematic review of normative literature. J Med Ethics. 2014;40(7):448–52. https://doi.org/10.1136/medethics-2013-101446

Brunton G, Oliver S, Thomas J. Innovations in framework synthesis as a systematic review method. Res Synthesis Methods. 2020;11(3):316–30. https://doi.org/10.1002/jrsm.1399

Berry NJ, Henry A, Danchin M, et al. When parents won’t vaccinate their children: a qualitative investigation of Australian primary care providers’ experiences. BMC Pediatr. 2017;17(1):19. https://doi.org/10.1186/s12887-017-0783-2

Helps C, Leask J, Barclay L, et al. Understanding non-vaccinating parents’ views to inform and improve clinical encounters: a qualitative study in an Australian community. BMJ Open. 2019;9(5):e026299. https://doi.org/10.1136/bmjopen-2018-026299

Wiley KE, Leask J, Attwell K, et al. Parenting and the vaccine refusal process: a new explanation of the relationship between lifestyle and vaccination trajectories. Soc Sci Med. 2020;263:113259. https://doi.org/10.1016/j.socscimed.2020.113259

Attwell K, Leask J, Meyer SB, et al. Vaccine rejecting parents’ Engagement with Expert systems that inform Vaccination Programs. J Bioeth Inq. 2017;14(1):65–76. https://doi.org/10.1007/s11673-016-9756-7 . [published Online First: 20161201].

Christou-Ergos M, Leask J, Wiley KE. How the experience of medical trauma shapes Australian non-vaccinating parents’ vaccine refusal for their children: a qualitative exploration. SSM - Qualitative Research in Health. 2022;2:100143. https://doi.org/10.1016/j.ssmqr.2022.100143

Rozbroj T, Lyons A, Lucke J. The mad leading the blind: perceptions of the vaccine-refusal movement among australians who support vaccination. Vaccine. 2019;37(40):5986–93. https://doi.org/10.1016/j.vaccine.2019.08.023 . [published Online First: 20190823].

Court J, Carter SM, Attwell K, et al. Labels matter: use and non-use of ‘anti-vax’ framing in Australian media discourse 2008–2018. Soc Sci Med. 2021;291:114502. https://doi.org/10.1016/j.socscimed.2021.114502

Wiley KE, Leask J, Attwell K, et al. Stigmatized for standing up for my child: a qualitative study of non-vaccinating parents in Australia. SSM Popul Health. 2021;16:100926. https://doi.org/10.1016/j.ssmph.2021.100926 . [published Online First: 20210916].

Attwell K, Drislane S. Australia’s ‘No jab no play’ policies: history, design and rationales. Aust N Z J Public Health. 2022;46(5):640–46. https://doi.org/10.1111/1753-6405.13289

Attwell K, Navin M. How policymakers employ ethical frames to design and introduce new policies: the case of childhood vaccine mandates in Australia. Policy and Politics. 2022;1–22. https://doi.org/10.1332/030557321X16476002878591

Wiley K, Robinson P, Degeling C, et al. Get your own house in order’: qualitative dialogue groups with nonvaccinating parents on how Measles outbreaks in their community should be managed. Health Expect. 2022;25(4):1678–90. https://doi.org/10.1111/hex.13511

Birchley G, Ives J. Fallacious, misleading and unhelpful: the case for removing ‘systematic review’ from bioethics nomenclature. Bioethics. 2022;36(6):635–47. https://doi.org/10.1111/bioe.13024

Yahya M. Polio vaccines—no thank you! Barriers to polio eradication in Northern Nigeria. Afr Affairs. 2007;106(423):185–204. https://doi.org/10.1093/afraf/adm016

Khattak FA, Rehman K, Shahzad M, et al. Prevalence of parental refusal rate and its associated factors in routine immunization by using WHO Vaccine Hesitancy tool: a Cross sectional study at district Bannu, KP, Pakistan. Int J Infect Dis. 2021;104:117–24. https://doi.org/10.1016/j.ijid.2020.12.029

Download references

This review was funded by the Australian National Health and Medical Research Council, grant number GNT1126543.

Author information

Authors and affiliations.

Sydney School of Public Health, The University of Sydney, Edward Ford Building A27, Sydney, 2006, Australia

Kerrie Wiley, Maria Christou-Ergos, Penelope Robinson & Catherine Helps

Australian Centre for Health Engagement, Evidence and Values, The University of Wollongong, Wollongong, 2522, Australia

Chris Degeling & Stacy M Carter

Melbourne School of Population and Global Health, The University of Melbourne, Melbourne, 3010, Australia

Rosalind McDougall

School of Social Sciences, Asian Studies & Politics, International Relations, University of Western Australia, Perth, 6009, Australia

Katie Attwell & Shevaun Drislane

You can also search for this author in PubMed   Google Scholar

Contributions

KW contributed to study design and search strategy development, ran the searches, managed the screening and inclusion process, screened articles for inclusion, extracted data, analysed and interpreted data and co-led manuscript drafting; MC ran updated searches, screened articles for inclusion and extracted data, assisted with analysis and interpretation and co-led manuscript drafting; CD contributed to study design and search strategy development, provided technical guidance, screened articles for inclusion and contributed to manuscript drafts; RM contributed to study design and search strategy development, provided technical guidance, screened articles for inclusion and contributed to manuscript drafts; PR screened articles for inclusion, extracted data, assisted with analysis and contributed to manuscript drafts; KA contributed to search strategy development, screened articles for inclusion and contributed to manuscript drafts; CH screened articles for inclusion and contributed to manuscript drafts; SD screened articles for inclusion and contributed to manuscript drafts; SMC contributed to study design and search strategy development, provided technical guidance, screened articles for inclusion and contributed to manuscript drafts.

Corresponding author

Correspondence to Kerrie Wiley .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Cite this article.

Wiley, K., Christou-Ergos, M., Degeling, C. et al. Childhood vaccine refusal and what to do about it: a systematic review of the ethical literature. BMC Med Ethics 24 , 96 (2023). https://doi.org/10.1186/s12910-023-00978-x

Download citation

Received : 20 February 2023

Accepted : 31 October 2023

Published : 08 November 2023

DOI : https://doi.org/10.1186/s12910-023-00978-x

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Vaccination
• Immunization
• Vaccine refusal
• Systematic review
• Normative literature
• Medical ethics

BMC Medical Ethics

ISSN: 1472-6939

• Submission enquiries: [email protected]
• General enquiries: [email protected]