- University of Texas Libraries
Steps in the literature review process.
- What is a literature review?
- Define your research question
- Determine inclusion and exclusion criteria
- Choose databases and search
- Review Results
- Synthesize Results
- Analyze Results
- Librarian Support
- You may need to some exploratory searching of the literature to get a sense of scope, to determine whether you need to narrow or broaden your focus
- Identify databases that provide the most relevant sources, and identify relevant terms (controlled vocabularies) to add to your search strategy
- Finalize your research question
- Think about relevant dates, geographies (and languages), methods, and conflicting points of view
- Conduct searches in the published literature via the identified databases
- Check to see if this topic has been covered in other discipline's databases
- Examine the citations of on-point articles for keywords, authors, and previous research (via references) and cited reference searching.
- Save your search results in a citation management tool (such as Zotero, Mendeley or EndNote)
- De-duplicate your search results
- Make sure that you've found the seminal pieces -- they have been cited many times, and their work is considered foundational
- Check with your professor or a librarian to make sure your search has been comprehensive
- Evaluate the strengths and weaknesses of individual sources and evaluate for bias, methodologies, and thoroughness
- Group your results in to an organizational structure that will support why your research needs to be done, or that provides the answer to your research question
- Develop your conclusions
- Are there gaps in the literature?
- Where has significant research taken place, and who has done it?
- Is there consensus or debate on this topic?
- Which methodological approaches work best?
- For example: Background, Current Practices, Critics and Proponents, Where/How this study will fit in
- Organize your citations and focus on your research question and pertinent studies
- Compile your bibliography
Note: The first four steps are the best points at which to contact a librarian. Your librarian can help you determine the best databases to use for your topic, assess scope, and formulate a search strategy.
Videos Tutorials about Literature Reviews
This 4.5 minute video from Academic Education Materials has a Creative Commons License and a British narrator.
- Last Updated: Oct 26, 2022 2:49 PM
- URL: https://guides.lib.utexas.edu/literaturereviews
Graduate Research: Guide to the Literature Review
- "Literature review" defined
- Research Communication Graphic
- Literature Review Steps
- Evaluating information
- Search techniques
- Citing Styles
- Ethical Use of Information
- Research Databases This link opens in a new window
- Get Full Text
- Reading a Scholarly Article
- Author Rights
- Selecting a publisher
Introduction to Research Process: Literature Review Steps
When seeking information for a literature review or for any purpose, it helps to understand information-seeking as a process that you can follow. 5 Each of the six (6) steps has its own section in this web page with more detail. Do (and re-do) the following six steps:
1. Define your topic. The first step is defining your task -- choosing a topic and noting the questions you have about the topic. This will provide a focus that guides your strategy in step II and will provide potential words to use in searches in step III.
2. Develop a strategy. Strategy involves figuring out where the information might be and identifying the best tools for finding those types of sources. The strategy section identifies specific types of research databases to use for specific purposes.
3. Locate the information . In this step, you implement the strategy developed in II in order to actually locate specific articles, books, technical reports, etc.
4. Use and Evaluate the information. Having located relevant and useful material, in step IV you read and analyze the items to determine whether they have value for your project and credibility as sources.
5. Synthesize. In step V, you will make sense of what you've learned and demonstrate your knowledge. You will thoroughly understand, organize and integrate the information --become knowledgeable-- so that you are able to use your own words to support and explain your research project and its relationship to existing research by others.
6. Evaluate your work. At every step along the way, you should evaluate your work. However, this final step is a last check to make sure your work is complete and of high quality.
Continue below to begin working through the process.
5. Eisenberg, M. B., & Berkowitz, R. E. (1990). Information Problem-Solving: the Big Six Skills Approach to Library & Information Skills Instruction . Norwood, NJ: Ablex Publishing.
1. Define your topic.
I. Define your topic
A. Many students have difficulty selecting a topic. You want to find a topic you find interesting and will enjoy learning more about.
B. Students often select a topic that is too broad. You may have a broad topic in mind initially and will need to narrow it.
1. To help narrow a broad topic :
1). Try this technique for brainstorming to narrow your focus.
a) Step 1. Write down your broad topic.
b) Step 2. Write down a "specific kind" or "specific aspect" of the topic you identified in step 1.
c) Step 3. Write down an aspect --such as an attribute or behavior-- of the "specific kind" you identified in step 2.
d) Step 4. Continue to add levels of specificity as needed to get to a focus that is manageable. However, you may want to begin researching the literature before narrowing further to give yourself the opportunity to explore what others are doing and how that might impact the direction that you take for your own research.
2) Three examples of using the narrowing technique. These examples start with very, very broad topics, so the topic at step 3 or 4 in these examples would be used for a preliminary search in the literature in order to identify a more specific focus. Greater specificity than level 3 or 4 will ultimately be necessary for developing a specific research question. And we may discover in our preliminary research that we need to alter the direction that we originally were taking.
a) Example 1.
Step 1. information security
Step 2. protocols
Step 3. handshake protocol
Brainstorming has brought us to focus on the handshake protocol.
b) Example 2.
Step 1. information security
Step 2. single sign-on authentication
Step 3. analyzing
Step 4. methods
Brainstorming has brought us to focus on methods for analyzing the security of single sign-on authentication
c) Example 3. The diagram below is an example using the broad topic of "software" to show two potential ways to begin to narrow the topic.
C. Once you have completed the brainstorming process and your topic is more focused, you can do preliminary research to help you identify a specific research question .
1) Examine overview sources such as subject-specific encyclopedias and textbooks that are likely to break down your specific topic into sub-topics and to highlight core issues that could serve as possible research questions. [See section II. below on developing a strategy to learn how to find these encyclopedias]
2). Search the broad topic in a research database that includes scholarly journals and professional magazines (to find technical and scholarly articles) and scan recent article titles for ideas. [See section II. below on developing a strategy to learn how to find trade and scholarly journal articles]
D. Once you have identified a research question or questions, ask yourself what you need to know to answer the questions. For example,
1. What new knowledge do I need to gain?
2. What has already been answered by prior research of other scholars?
E. Use the answers to the questions in C. to identify what words to use to describe the topic when you are doing searches.
1. Identify key words
a. For example , if you are investigating "security audits in banking", key terms to combine in your searches would be: security, audits, banking.
2. Create a list of alternative ways of referring to a key word or phrase
a.For example , "information assurance" may be referred to in various ways such as: "information assurance," "information security," and "computer security."
b. Use these alternatives when doing searches.
3. As you are searching, pay attention to how others are writing about the topic and add new words or phrases to your searches if appropriate.
2. Develop a strategy.
II. Develop a strategy for finding the information.
A. Start by considering what types of source might contain the information you need . Do you need a dictionary for definitions? a directory for an address? the history of a concept or technique that might be in a book or specialized encyclopedia? today's tech news in an online tech magazine or newspaper? current research in a journal article? background information that might be in a specialized encyclopedia? data or statistics from a specific organization or website? Note that you will typically have online access to these source types.
B. This section provides a description of some of the common types of information needed for research.
1. For technical and business analysis , look for articles in technical and trade magazines . These articles are written by information technology professionals to help other IT professionals do their jobs better. Content might include news on new developments in hardware or software, techniques, tools, and practical advice. Technical journals are also likely to have product ads relevant to information technology workers and to have job ads. Examples iof technical magazines include Network Computing and IEEE Spectrum .
2. To read original research studies , look for articles in scholarly journals and conference proceedings . They will provide articles written by information technology professionals who are reporting original research; that is, research that has been done by the authors and is being reported for the first time. The audience for original research articles is other information technology scholars and professionals. Examples of scholarly journals include Journal of Applied Security Research , Journal of Management Information Systems , IEEE Transactions on Computers , and ACM Transactions on Information and System Security .
3. For original research being reported to funding agencies , look for technical reports on agency websites. Technical reports are researcher reports to funding agencies about progress on or completion of research funded by the agency.
4. For in-depth, comprehensive information on a topic , look for book-length volumes . All chapters in the book might be written by the same author(s) or might be a collection of separate papers written by different authors.
5. To learn about an unfamiliar topic , use textbooks , specialized encyclopedias and handbooks to get get overviews of topics, history/background, and key issues explained.
6. For instructions for hardware, software, networking, etc., look for manuals that provide step-by-step instructions.
7. For technical details about inventions (devices, instruments, machines), look for patent documents .
C. NOTE - In order to search for and find original research studies, it will help if you understand how information is produced, packaged and communicated within your profession. This is explained in the tab "Research Communication: Graphic."
3. Locate the information.
III. Locate the information
A. Use search tools designed to find the sources you want. Types of sources were described in section II. above.
Always feel free to Ask a librarian for assistance when you have questions about where and how locate the information you need.
B. Evaluate the search results (no matter where you find the information)
1. Evaluate the items you find using at least these 5 criteria:
a. accuracy -- is the information reliable and error free?
1) Is there an editor or someone who verifies/checks the information?
2) Is there adequate documentation: bibliography, footnotes, credits?
3) Are the conclusions justified by the information presented?
b. authority -- is the source of the information reputable?
1) How did you find the source of information: an index to edited/peer-reviewed material, in a bibliography from a published article, etc.?
2) What type of source is it: sensationalistic, popular, scholarly?
c. objectivity -- does the information show bias?
1) What is the purpose of the information: to inform, persuade, explain, sway opinion, advertise?
2) Does the source show political or cultural biases?
d. currency -- is the information current? does it cover the time period you need?
e. coverage -- does it provide the evidence or information you need?
2. Is the search producing the material you need? -- the right content? the right quality? right time period? right geographical location? etc. If not, are you using
a. the right sources?
b. the right tools to get to the sources?
c. are you using the right words to describe the topic?
3. Have you discovered additional terms that should be searched? If so, search those terms.
4. Have you discovered additional questions you need to answer? If so, return to section A above to begin to answer new questions.
4. Use and evaluate the information.
IV. Use the information.
A. Read, hear or view the source
1. Evaluate: Does the material answer your question(s)? -- right content? If not, return to B.
2. Evaluate: Is the material appropriate? -- right quality? If not, return to B.
B. Extract the information from the source : copy/download information, take notes, record citation, keep track of items using a citation manager.
1. Note taking (these steps will help you when you begin to write your thesis and/or document your project.):
a. Write the keywords you use in your searches to avoid duplicating previous searches if you return to search a research database again. Keeping track of keywords used will also save you time if your search is interrupted or you need return and do the search again for some other reason. It will help you remember which search terms worked successfully in which databases
b. Write the citations or record the information needed to cite each article/document you plan to read and use, or make sure that any saved a copy of the article includes all the information needed to cite it. Some article pdf files may not include all of the information needed to cite, and it's a waste of your valuable time to have to go back to search and find the items again in order to be able to cite them. Using citation management software such as EndNote will help keep track of citations and help create bibliographies for your research papers.
c. Write a summary of each article you read and/or why you want to use it.
A. Organize and integrate information from multiple sources
B. Present the information (create report, speech, etc. that communicates)
C. Cite material using the style required by your professor or by the venue (conference, publication, etc.). For help with citation styles, see Guide to Citing Sources . A link to the citing guide is also available in the "Get Help" section on the left side of the Library home page
6. Evaluate your work.
VI. Evaluate the paper, speech, or whatever you are using to communicate your research.
A. Is it effective?
B. Does it meet the requirements?
C. Ask another student or colleague to provide constructive criticism of your paper/project.
- << Previous: Research Communication Graphic
- Next: Evaluating information >>
- Last Updated: Sep 25, 2023 9:43 AM
- URL: https://library.dsu.edu/graduate-research
- Learn How to Use the Library
- Providers & Employees
- Research Help
- All Research Guides
Conducting Literature Reviews
- About Literature Reviews
Step 1: the research question, step 2: search the literature, step 3: manage results, step 4: synthesize information, step 5: write the review.
- Additional Resources
- APA Style (7th ed.) This link opens in a new window
Like research, writing a literature review is an iterative process. Here is a very broad example of the process:
- Frame the research question and determine the scope of the literature review
- Search relevant bodies of literature
- Manage and organize search results
- Synthesize the literature
- Write an assessment of the literature
The initial steps should already be familiar to you, as they parallel steps of the research process you have used before.
Research questions, like topics, must be specific and focused so that you can 1) search for materials to address the question, and 2) write a literature review that is manageable in scope and purpose.
Developing a research question is the next logical step after selecting and then narrowing a topic. It is important to have a research question because it focuses your next step in the literature review process: searching. As Booth (2008) explains in The Craft of Research : "If a writer asks no specific question worth asking, he can offer no specific answer worth supporting. And without an answer to support, he cannot select from all the data he could find on a topic to just those relevant to his answer" (p. 41).
Once you have selected and narrowed your topic, ask yourself questions about the topic's:
- History (Is is part of a large context? What is its own internal history? How has it changed over time?)
- Structure and composition (Is it part of a larger system/structure? How do its parts fit together?)
- Categorization (Can you compare/contrast it with similar topics? Does it belong to a group of similar kinds?)
You can also:
- Turn positive questions into negative ones by focusing on "nots" (why didn't this happen? why isn't this significant in context?) or by contrasting differences
- Ask "what if" speculative questions (what if your topic disappeared? Was put in a different context?)
- Ask questions suggested by your initial background research, such as those that build on agreement (Author X made a persuasive point...) or reflect disagreement (Author Y's conclusion doesn't account for this contextual element...)
You may find that you need to reframe or revise your question as you continue through the literature process. That's ok! Remember, the literature review process is iterative.
For more detailed information on forming and evaluating research questions, see these books available to order through ILL from OhioLINK.
- OhioLINK Library Catalog This link opens in a new window Catalog of books and other materials held in Ohio college and university libraries.
- The Research Process Get help with selecting and narrowing a topic.
General guidance on where to search for sources:
- Where to Find Sources
Subject-specific guidance on where to search for sources:
- Evidence-Based Practice by Mike Jundi Last Updated Jan 26, 2023 53 views this year
- Finding Legislation, Data, & Statistics by Mike Jundi Last Updated May 23, 2023 18 views this year
- Nursing Research by Mike Jundi Last Updated Jan 26, 2023 86 views this year
- RAD 112 - Introduction to Radiography by Mike Jundi Last Updated Jan 26, 2023 21 views this year
- RAD 246 - Radiographic Pathology by Mike Jundi Last Updated May 4, 2023 32 views this year
- Social Work Resources by Mike Jundi Last Updated Dec 3, 2021 3 views this year
How to search for sources by developing a search strategy:
- How to Search for Sources
General guidance on using catalogs and databases:
- Basic Library Tutorials by Mike Jundi Last Updated Sep 28, 2023 644 views this year
Research management involves collecting, organizing, and citing.
Research management is also based largely on personal preference. Do you have a system that works for you? Great! If you aren't used to research management and/or don't have an effective system in place, you have options.
- Do-it-yourself: maintain your resources on your computer's hard drive or on the cloud (Microsoft OneDrive, Google Drive, DropBox)
- Use a free research management software (Mendeley, Zotero, EndNote)
Regardless of what system you use, it is necessary to keep track of the these elements:
- The literature you found (Did you find full text in a PDF? Save it. Did you find a record in a database, but need to request the article? Save the permalink to the record.)
- The full APA citation for the literature
- An easy way to track results you've found in databases is to create folders
Finally, you will need a note-taking system that will help you record the key concepts from the literature when you read and synthesize it. If you already have one, great! If you struggle with note-taking, see the links below.
What is synthesis?
Synthesizing information is much the opposite of analyzing information. When you read an article or book, you have to pull out specific concepts from the larger document in order to understand it. This is analyzing.
When you synthesize information, you take specific concepts and consider them together to understand how they compare/contrast and how they relate to one another. In other terms, synthesis involves combining multiple elements to create a whole. In regard to literature reviews, the elements refer to the findings from the literature you've gathered. The whole then becomes your conclusion(s) about those findings.
How do I synthesize information?
Note: This stage in the literature review process is as iterative and personal as any other. These steps offer a guideline, but do what works for you best.
- This is where you really decide if you want to read specific materials
- If you have gathered a substantial amount of literature and reading all of it would prove overwhelming, read the abstracts to get a better idea of the content, then select the materials that would best support your review
- Describe and analyze the findings (What were the results? How did the authors get these results? What are the impacts? Etc.)
- Identify the key concepts
- Compare and contrast findings, concepts, conclusions, methods, etc.
- Evaluate the quality and significance of findings, concepts, conclusions, methods, etc.
- Interpret the findings, concepts, conclusions, methods, etc. in the context of your research question
- This is the step where your synthesis of the information will lead to logical conclusions about that information
- These conclusions should speak directly to your research question (i.e. your question should have an answer)
Visit the link below for helpful resources on note-taking:
- Other Helpful Tips: Note-Taking & Proofreading
You are expected to follow APA Style in your writing. Visit this guide for an introduction, tips, and tutorials:
- APA Style Resources (7th ed.) by Mike Jundi Last Updated Jan 13, 2023 311 views this year
The structure and flow of your literature review should be logical and should reflect the synthesis you have done.
A common pitfall for students is using an author-driven structure , which might look something like this:
- Author 1 says x
- Author 2 says y
- Author ∞ says...
Why doesn't the author-driven structure work?
- Leans toward listing or summarizing information
- Doesn't illustrate synthesis of information (all of the findings are listed based on where they came from, not their meaning, impact, or significance)
What structures do work? The APA suggests three structures for literature reviews:
- Theme-based (group studies based on common themes or concepts present)
- Methodology-based (group studies based on the methodologies used)
- Chronological (group studies based on the historical developments in the field)
The theme-based structure is applicable to most bodies of literature you might gather. It may look like this:
- Concept x from author 1
- Concept a from author 5
- Concept y from author 2
Why does the them-based structure work better?
- It avoids listing information
- It clearly shows the synthesis that occurred
- It illustrates the connections between concepts and the significance of particular concepts
- << Previous: About Literature Reviews
- Next: Additional Resources >>
- Last Updated: Oct 13, 2022 3:08 PM
- URL: https://aultman.libguides.com/literaturereviews
Aultman Health Sciences Library
Aultman Education Center, C2-230, 2600 Sixth St SW, Canton, OH 44710 | 330-363-5000 | [email protected]
Write a literature review.
- Examples and Further Information
Not to be confused with a book review, a literature review surveys scholarly articles, books and other sources (e.g. dissertations, conference proceedings) relevant to a particular issue, area of research, or theory, providing a description, summary, and critical evaluation of each work. The purpose is to offer an overview of significant literature published on a topic.
Similar to primary research, development of the literature review requires four stages:
- Problem formulation—which topic or field is being examined and what are its component issues?
- Literature search—finding materials relevant to the subject being explored
- Data evaluation—determining which literature makes a significant contribution to the understanding of the topic
- Analysis and interpretation—discussing the findings and conclusions of pertinent literature
Literature reviews should comprise the following elements:
- An overview of the subject, issue or theory under consideration, along with the objectives of the literature review
- Division of works under review into categories (e.g. those in support of a particular position, those against, and those offering alternative theses entirely)
- Explanation of how each work is similar to and how it varies from the others
- Conclusions as to which pieces are best considered in their argument, are most convincing of their opinions, and make the greatest contribution to the understanding and development of their area of research
In assessing each piece, consideration should be given to:
- Provenance—What are the author's credentials? Are the author's arguments supported by evidence (e.g. primary historical material, case studies, narratives, statistics, recent scientific findings)?
- Objectivity—Is the author's perspective even-handed or prejudicial? Is contrary data considered or is certain pertinent information ignored to prove the author's point?
- Persuasiveness—Which of the author's theses are most/least convincing?
- Value—Are the author's arguments and conclusions convincing? Does the work ultimately contribute in any significant way to an understanding of the subject?
3. Definition and Use/Purpose
A literature review may constitute an essential chapter of a thesis or dissertation, or may be a self-contained review of writings on a subject. In either case, its purpose is to:
- Place each work in the context of its contribution to the understanding of the subject under review
- Describe the relationship of each work to the others under consideration
- Identify new ways to interpret, and shed light on any gaps in, previous research
- Resolve conflicts amongst seemingly contradictory previous studies
- Identify areas of prior scholarship to prevent duplication of effort
- Point the way forward for further research
- Place one's original work (in the case of theses or dissertations) in the context of existing literature
The literature review itself, however, does not present new primary scholarship.
- Next: Examples and Further Information >>
Creative Commons Attribution 3.0 License except where otherwise noted.
The land on which we gather is the unceded territory of the Awaswas-speaking Uypi Tribe. The Amah Mutsun Tribal Band, comprised of the descendants of indigenous people taken to missions Santa Cruz and San Juan Bautista during Spanish colonization of the Central Coast, is today working hard to restore traditional stewardship practices on these lands and heal from historical trauma.
The land acknowledgement used at UC Santa Cruz was developed in partnership with the Amah Mutsun Tribal Band Chairman and the Amah Mutsun Relearning Program at the UCSC Arboretum .
Planning and carrying out a literature review
http://prezi.com/w9f6pi15hyyb/?utm_campaign=share&utm_medium=copy Draft Text
4. Stages of a literature review
Before you start your literature review, you should already have defined your research topic . Familiarise yourself with your topic's main themes, and be aware of your limits .
There are four main steps you’ll need to take to complete your literature review:
1. You will firstly need to find literature which is relevant to your research topic. Before starting your search, make sure that you have defined your subject . Your literature search should be led by the main themes and limits of your research.
2. Evaluating your results is the next step. You will need to make a decision as to whether the literature you have found makes a significant contribution to the understanding of your topic. The Prezi ( You've finished your literature search... What next? ) at the bottom of the page will guide you through the process of deciding what literature needs to be included in your literature review.
3. You will then need to analyse and interpret the literature you have decided to include in your review. Identify themes/ideas/theories/approaches that have emerged from reading the literature.
4. Once you are satisfied that you have reviewed enough literature relevant to your research topic, you can get down to writing . You will need to introduce and explain each theme (or theory/approach), provide evidence from the literature, comment critically on the literature and relate it to your own research .
Prezi: You've finished your literature search... What next?
Whilst this process is usually followed by medical students undertaking a systematic review, the framework could easily be used by students from any discipline.
The Teaching & Learning Support Team are always ready to assist: Contact the T&LS Team
An official website of the United States government
The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.
The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.
- Account settings
- Browse Titles
NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.
Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.
Handbook of eHealth Evaluation: An Evidence-based Approach [Internet].
Chapter 9 methods for literature reviews.
Guy Paré and Spyros Kitsiou .
Literature reviews play a critical role in scholarship because science remains, first and foremost, a cumulative endeavour ( vom Brocke et al., 2009 ). As in any academic discipline, rigorous knowledge syntheses are becoming indispensable in keeping up with an exponentially growing eHealth literature, assisting practitioners, academics, and graduate students in finding, evaluating, and synthesizing the contents of many empirical and conceptual papers. Among other methods, literature reviews are essential for: (a) identifying what has been written on a subject or topic; (b) determining the extent to which a specific research area reveals any interpretable trends or patterns; (c) aggregating empirical findings related to a narrow research question to support evidence-based practice; (d) generating new frameworks and theories; and (e) identifying topics or questions requiring more investigation ( Paré, Trudel, Jaana, & Kitsiou, 2015 ).
Literature reviews can take two major forms. The most prevalent one is the “literature review” or “background” section within a journal paper or a chapter in a graduate thesis. This section synthesizes the extant literature and usually identifies the gaps in knowledge that the empirical study addresses ( Sylvester, Tate, & Johnstone, 2013 ). It may also provide a theoretical foundation for the proposed study, substantiate the presence of the research problem, justify the research as one that contributes something new to the cumulated knowledge, or validate the methods and approaches for the proposed study ( Hart, 1998 ; Levy & Ellis, 2006 ).
The second form of literature review, which is the focus of this chapter, constitutes an original and valuable work of research in and of itself ( Paré et al., 2015 ). Rather than providing a base for a researcher’s own work, it creates a solid starting point for all members of the community interested in a particular area or topic ( Mulrow, 1987 ). The so-called “review article” is a journal-length paper which has an overarching purpose to synthesize the literature in a field, without collecting or analyzing any primary data ( Green, Johnson, & Adams, 2006 ).
When appropriately conducted, review articles represent powerful information sources for practitioners looking for state-of-the art evidence to guide their decision-making and work practices ( Paré et al., 2015 ). Further, high-quality reviews become frequently cited pieces of work which researchers seek out as a first clear outline of the literature when undertaking empirical studies ( Cooper, 1988 ; Rowe, 2014 ). Scholars who track and gauge the impact of articles have found that review papers are cited and downloaded more often than any other type of published article ( Cronin, Ryan, & Coughlan, 2008 ; Montori, Wilczynski, Morgan, Haynes, & Hedges, 2003 ; Patsopoulos, Analatos, & Ioannidis, 2005 ). The reason for their popularity may be the fact that reading the review enables one to have an overview, if not a detailed knowledge of the area in question, as well as references to the most useful primary sources ( Cronin et al., 2008 ). Although they are not easy to conduct, the commitment to complete a review article provides a tremendous service to one’s academic community ( Paré et al., 2015 ; Petticrew & Roberts, 2006 ). Most, if not all, peer-reviewed journals in the fields of medical informatics publish review articles of some type.
The main objectives of this chapter are fourfold: (a) to provide an overview of the major steps and activities involved in conducting a stand-alone literature review; (b) to describe and contrast the different types of review articles that can contribute to the eHealth knowledge base; (c) to illustrate each review type with one or two examples from the eHealth literature; and (d) to provide a series of recommendations for prospective authors of review articles in this domain.
9.2. Overview of the Literature Review Process and Steps
As explained in Templier and Paré (2015) , there are six generic steps involved in conducting a review article:
- formulating the research question(s) and objective(s),
- searching the extant literature,
- screening for inclusion,
- assessing the quality of primary studies,
- extracting data, and
- analyzing data.
Although these steps are presented here in sequential order, one must keep in mind that the review process can be iterative and that many activities can be initiated during the planning stage and later refined during subsequent phases ( Finfgeld-Connett & Johnson, 2013 ; Kitchenham & Charters, 2007 ).
Formulating the research question(s) and objective(s): As a first step, members of the review team must appropriately justify the need for the review itself ( Petticrew & Roberts, 2006 ), identify the review’s main objective(s) ( Okoli & Schabram, 2010 ), and define the concepts or variables at the heart of their synthesis ( Cooper & Hedges, 2009 ; Webster & Watson, 2002 ). Importantly, they also need to articulate the research question(s) they propose to investigate ( Kitchenham & Charters, 2007 ). In this regard, we concur with Jesson, Matheson, and Lacey (2011) that clearly articulated research questions are key ingredients that guide the entire review methodology; they underscore the type of information that is needed, inform the search for and selection of relevant literature, and guide or orient the subsequent analysis. Searching the extant literature: The next step consists of searching the literature and making decisions about the suitability of material to be considered in the review ( Cooper, 1988 ). There exist three main coverage strategies. First, exhaustive coverage means an effort is made to be as comprehensive as possible in order to ensure that all relevant studies, published and unpublished, are included in the review and, thus, conclusions are based on this all-inclusive knowledge base. The second type of coverage consists of presenting materials that are representative of most other works in a given field or area. Often authors who adopt this strategy will search for relevant articles in a small number of top-tier journals in a field ( Paré et al., 2015 ). In the third strategy, the review team concentrates on prior works that have been central or pivotal to a particular topic. This may include empirical studies or conceptual papers that initiated a line of investigation, changed how problems or questions were framed, introduced new methods or concepts, or engendered important debate ( Cooper, 1988 ). Screening for inclusion: The following step consists of evaluating the applicability of the material identified in the preceding step ( Levy & Ellis, 2006 ; vom Brocke et al., 2009 ). Once a group of potential studies has been identified, members of the review team must screen them to determine their relevance ( Petticrew & Roberts, 2006 ). A set of predetermined rules provides a basis for including or excluding certain studies. This exercise requires a significant investment on the part of researchers, who must ensure enhanced objectivity and avoid biases or mistakes. As discussed later in this chapter, for certain types of reviews there must be at least two independent reviewers involved in the screening process and a procedure to resolve disagreements must also be in place ( Liberati et al., 2009 ; Shea et al., 2009 ). Assessing the quality of primary studies: In addition to screening material for inclusion, members of the review team may need to assess the scientific quality of the selected studies, that is, appraise the rigour of the research design and methods. Such formal assessment, which is usually conducted independently by at least two coders, helps members of the review team refine which studies to include in the final sample, determine whether or not the differences in quality may affect their conclusions, or guide how they analyze the data and interpret the findings ( Petticrew & Roberts, 2006 ). Ascribing quality scores to each primary study or considering through domain-based evaluations which study components have or have not been designed and executed appropriately makes it possible to reflect on the extent to which the selected study addresses possible biases and maximizes validity ( Shea et al., 2009 ). Extracting data: The following step involves gathering or extracting applicable information from each primary study included in the sample and deciding what is relevant to the problem of interest ( Cooper & Hedges, 2009 ). Indeed, the type of data that should be recorded mainly depends on the initial research questions ( Okoli & Schabram, 2010 ). However, important information may also be gathered about how, when, where and by whom the primary study was conducted, the research design and methods, or qualitative/quantitative results ( Cooper & Hedges, 2009 ). Analyzing and synthesizing data : As a final step, members of the review team must collate, summarize, aggregate, organize, and compare the evidence extracted from the included studies. The extracted data must be presented in a meaningful way that suggests a new contribution to the extant literature ( Jesson et al., 2011 ). Webster and Watson (2002) warn researchers that literature reviews should be much more than lists of papers and should provide a coherent lens to make sense of extant knowledge on a given topic. There exist several methods and techniques for synthesizing quantitative (e.g., frequency analysis, meta-analysis) and qualitative (e.g., grounded theory, narrative analysis, meta-ethnography) evidence ( Dixon-Woods, Agarwal, Jones, Young, & Sutton, 2005 ; Thomas & Harden, 2008 ).
9.3. Types of Review Articles and Brief Illustrations
EHealth researchers have at their disposal a number of approaches and methods for making sense out of existing literature, all with the purpose of casting current research findings into historical contexts or explaining contradictions that might exist among a set of primary research studies conducted on a particular topic. Our classification scheme is largely inspired from Paré and colleagues’ (2015) typology. Below we present and illustrate those review types that we feel are central to the growth and development of the eHealth domain.
9.3.1. Narrative Reviews
The narrative review is the “traditional” way of reviewing the extant literature and is skewed towards a qualitative interpretation of prior knowledge ( Sylvester et al., 2013 ). Put simply, a narrative review attempts to summarize or synthesize what has been written on a particular topic but does not seek generalization or cumulative knowledge from what is reviewed ( Davies, 2000 ; Green et al., 2006 ). Instead, the review team often undertakes the task of accumulating and synthesizing the literature to demonstrate the value of a particular point of view ( Baumeister & Leary, 1997 ). As such, reviewers may selectively ignore or limit the attention paid to certain studies in order to make a point. In this rather unsystematic approach, the selection of information from primary articles is subjective, lacks explicit criteria for inclusion and can lead to biased interpretations or inferences ( Green et al., 2006 ). There are several narrative reviews in the particular eHealth domain, as in all fields, which follow such an unstructured approach ( Silva et al., 2015 ; Paul et al., 2015 ).
Despite these criticisms, this type of review can be very useful in gathering together a volume of literature in a specific subject area and synthesizing it. As mentioned above, its primary purpose is to provide the reader with a comprehensive background for understanding current knowledge and highlighting the significance of new research ( Cronin et al., 2008 ). Faculty like to use narrative reviews in the classroom because they are often more up to date than textbooks, provide a single source for students to reference, and expose students to peer-reviewed literature ( Green et al., 2006 ). For researchers, narrative reviews can inspire research ideas by identifying gaps or inconsistencies in a body of knowledge, thus helping researchers to determine research questions or formulate hypotheses. Importantly, narrative reviews can also be used as educational articles to bring practitioners up to date with certain topics of issues ( Green et al., 2006 ).
Recently, there have been several efforts to introduce more rigour in narrative reviews that will elucidate common pitfalls and bring changes into their publication standards. Information systems researchers, among others, have contributed to advancing knowledge on how to structure a “traditional” review. For instance, Levy and Ellis (2006) proposed a generic framework for conducting such reviews. Their model follows the systematic data processing approach comprised of three steps, namely: (a) literature search and screening; (b) data extraction and analysis; and (c) writing the literature review. They provide detailed and very helpful instructions on how to conduct each step of the review process. As another methodological contribution, vom Brocke et al. (2009) offered a series of guidelines for conducting literature reviews, with a particular focus on how to search and extract the relevant body of knowledge. Last, Bandara, Miskon, and Fielt (2011) proposed a structured, predefined and tool-supported method to identify primary studies within a feasible scope, extract relevant content from identified articles, synthesize and analyze the findings, and effectively write and present the results of the literature review. We highly recommend that prospective authors of narrative reviews consult these useful sources before embarking on their work.
Darlow and Wen (2015) provide a good example of a highly structured narrative review in the eHealth field. These authors synthesized published articles that describe the development process of mobile health ( m-health ) interventions for patients’ cancer care self-management. As in most narrative reviews, the scope of the research questions being investigated is broad: (a) how development of these systems are carried out; (b) which methods are used to investigate these systems; and (c) what conclusions can be drawn as a result of the development of these systems. To provide clear answers to these questions, a literature search was conducted on six electronic databases and Google Scholar . The search was performed using several terms and free text words, combining them in an appropriate manner. Four inclusion and three exclusion criteria were utilized during the screening process. Both authors independently reviewed each of the identified articles to determine eligibility and extract study information. A flow diagram shows the number of studies identified, screened, and included or excluded at each stage of study selection. In terms of contributions, this review provides a series of practical recommendations for m-health intervention development.
9.3.2. Descriptive or Mapping Reviews
The primary goal of a descriptive review is to determine the extent to which a body of knowledge in a particular research topic reveals any interpretable pattern or trend with respect to pre-existing propositions, theories, methodologies or findings ( King & He, 2005 ; Paré et al., 2015 ). In contrast with narrative reviews, descriptive reviews follow a systematic and transparent procedure, including searching, screening and classifying studies ( Petersen, Vakkalanka, & Kuzniarz, 2015 ). Indeed, structured search methods are used to form a representative sample of a larger group of published works ( Paré et al., 2015 ). Further, authors of descriptive reviews extract from each study certain characteristics of interest, such as publication year, research methods, data collection techniques, and direction or strength of research outcomes (e.g., positive, negative, or non-significant) in the form of frequency analysis to produce quantitative results ( Sylvester et al., 2013 ). In essence, each study included in a descriptive review is treated as the unit of analysis and the published literature as a whole provides a database from which the authors attempt to identify any interpretable trends or draw overall conclusions about the merits of existing conceptualizations, propositions, methods or findings ( Paré et al., 2015 ). In doing so, a descriptive review may claim that its findings represent the state of the art in a particular domain ( King & He, 2005 ).
In the fields of health sciences and medical informatics, reviews that focus on examining the range, nature and evolution of a topic area are described by Anderson, Allen, Peckham, and Goodwin (2008) as mapping reviews . Like descriptive reviews, the research questions are generic and usually relate to publication patterns and trends. There is no preconceived plan to systematically review all of the literature although this can be done. Instead, researchers often present studies that are representative of most works published in a particular area and they consider a specific time frame to be mapped.
An example of this approach in the eHealth domain is offered by DeShazo, Lavallie, and Wolf (2009). The purpose of this descriptive or mapping review was to characterize publication trends in the medical informatics literature over a 20-year period (1987 to 2006). To achieve this ambitious objective, the authors performed a bibliometric analysis of medical informatics citations indexed in medline using publication trends, journal frequencies, impact factors, Medical Subject Headings (MeSH) term frequencies, and characteristics of citations. Findings revealed that there were over 77,000 medical informatics articles published during the covered period in numerous journals and that the average annual growth rate was 12%. The MeSH term analysis also suggested a strong interdisciplinary trend. Finally, average impact scores increased over time with two notable growth periods. Overall, patterns in research outputs that seem to characterize the historic trends and current components of the field of medical informatics suggest it may be a maturing discipline (DeShazo et al., 2009).
9.3.3. Scoping Reviews
Scoping reviews attempt to provide an initial indication of the potential size and nature of the extant literature on an emergent topic (Arksey & O’Malley, 2005; Daudt, van Mossel, & Scott, 2013 ; Levac, Colquhoun, & O’Brien, 2010). A scoping review may be conducted to examine the extent, range and nature of research activities in a particular area, determine the value of undertaking a full systematic review (discussed next), or identify research gaps in the extant literature ( Paré et al., 2015 ). In line with their main objective, scoping reviews usually conclude with the presentation of a detailed research agenda for future works along with potential implications for both practice and research.
Unlike narrative and descriptive reviews, the whole point of scoping the field is to be as comprehensive as possible, including grey literature (Arksey & O’Malley, 2005). Inclusion and exclusion criteria must be established to help researchers eliminate studies that are not aligned with the research questions. It is also recommended that at least two independent coders review abstracts yielded from the search strategy and then the full articles for study selection ( Daudt et al., 2013 ). The synthesized evidence from content or thematic analysis is relatively easy to present in tabular form (Arksey & O’Malley, 2005; Thomas & Harden, 2008 ).
One of the most highly cited scoping reviews in the eHealth domain was published by Archer, Fevrier-Thomas, Lokker, McKibbon, and Straus (2011) . These authors reviewed the existing literature on personal health record ( phr ) systems including design, functionality, implementation, applications, outcomes, and benefits. Seven databases were searched from 1985 to March 2010. Several search terms relating to phr s were used during this process. Two authors independently screened titles and abstracts to determine inclusion status. A second screen of full-text articles, again by two independent members of the research team, ensured that the studies described phr s. All in all, 130 articles met the criteria and their data were extracted manually into a database. The authors concluded that although there is a large amount of survey, observational, cohort/panel, and anecdotal evidence of phr benefits and satisfaction for patients, more research is needed to evaluate the results of phr implementations. Their in-depth analysis of the literature signalled that there is little solid evidence from randomized controlled trials or other studies through the use of phr s. Hence, they suggested that more research is needed that addresses the current lack of understanding of optimal functionality and usability of these systems, and how they can play a beneficial role in supporting patient self-management ( Archer et al., 2011 ).
9.3.4. Forms of Aggregative Reviews
Healthcare providers, practitioners, and policy-makers are nowadays overwhelmed with large volumes of information, including research-based evidence from numerous clinical trials and evaluation studies, assessing the effectiveness of health information technologies and interventions ( Ammenwerth & de Keizer, 2004 ; Deshazo et al., 2009 ). It is unrealistic to expect that all these disparate actors will have the time, skills, and necessary resources to identify the available evidence in the area of their expertise and consider it when making decisions. Systematic reviews that involve the rigorous application of scientific strategies aimed at limiting subjectivity and bias (i.e., systematic and random errors) can respond to this challenge.
Systematic reviews attempt to aggregate, appraise, and synthesize in a single source all empirical evidence that meet a set of previously specified eligibility criteria in order to answer a clearly formulated and often narrow research question on a particular topic of interest to support evidence-based practice ( Liberati et al., 2009 ). They adhere closely to explicit scientific principles ( Liberati et al., 2009 ) and rigorous methodological guidelines (Higgins & Green, 2008) aimed at reducing random and systematic errors that can lead to deviations from the truth in results or inferences. The use of explicit methods allows systematic reviews to aggregate a large body of research evidence, assess whether effects or relationships are in the same direction and of the same general magnitude, explain possible inconsistencies between study results, and determine the strength of the overall evidence for every outcome of interest based on the quality of included studies and the general consistency among them ( Cook, Mulrow, & Haynes, 1997 ). The main procedures of a systematic review involve:
- Formulating a review question and developing a search strategy based on explicit inclusion criteria for the identification of eligible studies (usually described in the context of a detailed review protocol).
- Searching for eligible studies using multiple databases and information sources, including grey literature sources, without any language restrictions.
- Selecting studies, extracting data, and assessing risk of bias in a duplicate manner using two independent reviewers to avoid random or systematic errors in the process.
- Analyzing data using quantitative or qualitative methods.
- Presenting results in summary of findings tables.
- Interpreting results and drawing conclusions.
Many systematic reviews, but not all, use statistical methods to combine the results of independent studies into a single quantitative estimate or summary effect size. Known as meta-analyses , these reviews use specific data extraction and statistical techniques (e.g., network, frequentist, or Bayesian meta-analyses) to calculate from each study by outcome of interest an effect size along with a confidence interval that reflects the degree of uncertainty behind the point estimate of effect ( Borenstein, Hedges, Higgins, & Rothstein, 2009 ; Deeks, Higgins, & Altman, 2008 ). Subsequently, they use fixed or random-effects analysis models to combine the results of the included studies, assess statistical heterogeneity, and calculate a weighted average of the effect estimates from the different studies, taking into account their sample sizes. The summary effect size is a value that reflects the average magnitude of the intervention effect for a particular outcome of interest or, more generally, the strength of a relationship between two variables across all studies included in the systematic review. By statistically combining data from multiple studies, meta-analyses can create more precise and reliable estimates of intervention effects than those derived from individual studies alone, when these are examined independently as discrete sources of information.
The review by Gurol-Urganci, de Jongh, Vodopivec-Jamsek, Atun, and Car (2013) on the effects of mobile phone messaging reminders for attendance at healthcare appointments is an illustrative example of a high-quality systematic review with meta-analysis. Missed appointments are a major cause of inefficiency in healthcare delivery with substantial monetary costs to health systems. These authors sought to assess whether mobile phone-based appointment reminders delivered through Short Message Service ( sms ) or Multimedia Messaging Service ( mms ) are effective in improving rates of patient attendance and reducing overall costs. To this end, they conducted a comprehensive search on multiple databases using highly sensitive search strategies without language or publication-type restrictions to identify all rct s that are eligible for inclusion. In order to minimize the risk of omitting eligible studies not captured by the original search, they supplemented all electronic searches with manual screening of trial registers and references contained in the included studies. Study selection, data extraction, and risk of bias assessments were performed independently by two coders using standardized methods to ensure consistency and to eliminate potential errors. Findings from eight rct s involving 6,615 participants were pooled into meta-analyses to calculate the magnitude of effects that mobile text message reminders have on the rate of attendance at healthcare appointments compared to no reminders and phone call reminders.
Meta-analyses are regarded as powerful tools for deriving meaningful conclusions. However, there are situations in which it is neither reasonable nor appropriate to pool studies together using meta-analytic methods simply because there is extensive clinical heterogeneity between the included studies or variation in measurement tools, comparisons, or outcomes of interest. In these cases, systematic reviews can use qualitative synthesis methods such as vote counting, content analysis, classification schemes and tabulations, as an alternative approach to narratively synthesize the results of the independent studies included in the review. This form of review is known as qualitative systematic review.
A rigorous example of one such review in the eHealth domain is presented by Mickan, Atherton, Roberts, Heneghan, and Tilson (2014) on the use of handheld computers by healthcare professionals and their impact on access to information and clinical decision-making. In line with the methodological guidelines for systematic reviews, these authors: (a) developed and registered with prospero ( www.crd.york.ac.uk/ prospero / ) an a priori review protocol; (b) conducted comprehensive searches for eligible studies using multiple databases and other supplementary strategies (e.g., forward searches); and (c) subsequently carried out study selection, data extraction, and risk of bias assessments in a duplicate manner to eliminate potential errors in the review process. Heterogeneity between the included studies in terms of reported outcomes and measures precluded the use of meta-analytic methods. To this end, the authors resorted to using narrative analysis and synthesis to describe the effectiveness of handheld computers on accessing information for clinical knowledge, adherence to safety and clinical quality guidelines, and diagnostic decision-making.
In recent years, the number of systematic reviews in the field of health informatics has increased considerably. Systematic reviews with discordant findings can cause great confusion and make it difficult for decision-makers to interpret the review-level evidence ( Moher, 2013 ). Therefore, there is a growing need for appraisal and synthesis of prior systematic reviews to ensure that decision-making is constantly informed by the best available accumulated evidence. Umbrella reviews , also known as overviews of systematic reviews, are tertiary types of evidence synthesis that aim to accomplish this; that is, they aim to compare and contrast findings from multiple systematic reviews and meta-analyses ( Becker & Oxman, 2008 ). Umbrella reviews generally adhere to the same principles and rigorous methodological guidelines used in systematic reviews. However, the unit of analysis in umbrella reviews is the systematic review rather than the primary study ( Becker & Oxman, 2008 ). Unlike systematic reviews that have a narrow focus of inquiry, umbrella reviews focus on broader research topics for which there are several potential interventions ( Smith, Devane, Begley, & Clarke, 2011 ). A recent umbrella review on the effects of home telemonitoring interventions for patients with heart failure critically appraised, compared, and synthesized evidence from 15 systematic reviews to investigate which types of home telemonitoring technologies and forms of interventions are more effective in reducing mortality and hospital admissions ( Kitsiou, Paré, & Jaana, 2015 ).
9.3.5. Realist Reviews
Realist reviews are theory-driven interpretative reviews developed to inform, enhance, or supplement conventional systematic reviews by making sense of heterogeneous evidence about complex interventions applied in diverse contexts in a way that informs policy decision-making ( Greenhalgh, Wong, Westhorp, & Pawson, 2011 ). They originated from criticisms of positivist systematic reviews which centre on their “simplistic” underlying assumptions ( Oates, 2011 ). As explained above, systematic reviews seek to identify causation. Such logic is appropriate for fields like medicine and education where findings of randomized controlled trials can be aggregated to see whether a new treatment or intervention does improve outcomes. However, many argue that it is not possible to establish such direct causal links between interventions and outcomes in fields such as social policy, management, and information systems where for any intervention there is unlikely to be a regular or consistent outcome ( Oates, 2011 ; Pawson, 2006 ; Rousseau, Manning, & Denyer, 2008 ).
To circumvent these limitations, Pawson, Greenhalgh, Harvey, and Walshe (2005) have proposed a new approach for synthesizing knowledge that seeks to unpack the mechanism of how “complex interventions” work in particular contexts. The basic research question — what works? — which is usually associated with systematic reviews changes to: what is it about this intervention that works, for whom, in what circumstances, in what respects and why? Realist reviews have no particular preference for either quantitative or qualitative evidence. As a theory-building approach, a realist review usually starts by articulating likely underlying mechanisms and then scrutinizes available evidence to find out whether and where these mechanisms are applicable ( Shepperd et al., 2009 ). Primary studies found in the extant literature are viewed as case studies which can test and modify the initial theories ( Rousseau et al., 2008 ).
The main objective pursued in the realist review conducted by Otte-Trojel, de Bont, Rundall, and van de Klundert (2014) was to examine how patient portals contribute to health service delivery and patient outcomes. The specific goals were to investigate how outcomes are produced and, most importantly, how variations in outcomes can be explained. The research team started with an exploratory review of background documents and research studies to identify ways in which patient portals may contribute to health service delivery and patient outcomes. The authors identified six main ways which represent “educated guesses” to be tested against the data in the evaluation studies. These studies were identified through a formal and systematic search in four databases between 2003 and 2013. Two members of the research team selected the articles using a pre-established list of inclusion and exclusion criteria and following a two-step procedure. The authors then extracted data from the selected articles and created several tables, one for each outcome category. They organized information to bring forward those mechanisms where patient portals contribute to outcomes and the variation in outcomes across different contexts.
9.3.6. Critical Reviews
Lastly, critical reviews aim to provide a critical evaluation and interpretive analysis of existing literature on a particular topic of interest to reveal strengths, weaknesses, contradictions, controversies, inconsistencies, and/or other important issues with respect to theories, hypotheses, research methods or results ( Baumeister & Leary, 1997 ; Kirkevold, 1997 ). Unlike other review types, critical reviews attempt to take a reflective account of the research that has been done in a particular area of interest, and assess its credibility by using appraisal instruments or critical interpretive methods. In this way, critical reviews attempt to constructively inform other scholars about the weaknesses of prior research and strengthen knowledge development by giving focus and direction to studies for further improvement ( Kirkevold, 1997 ).
Kitsiou, Paré, and Jaana (2013) provide an example of a critical review that assessed the methodological quality of prior systematic reviews of home telemonitoring studies for chronic patients. The authors conducted a comprehensive search on multiple databases to identify eligible reviews and subsequently used a validated instrument to conduct an in-depth quality appraisal. Results indicate that the majority of systematic reviews in this particular area suffer from important methodological flaws and biases that impair their internal validity and limit their usefulness for clinical and decision-making purposes. To this end, they provide a number of recommendations to strengthen knowledge development towards improving the design and execution of future reviews on home telemonitoring.
Table 9.1 outlines the main types of literature reviews that were described in the previous sub-sections and summarizes the main characteristics that distinguish one review type from another. It also includes key references to methodological guidelines and useful sources that can be used by eHealth scholars and researchers for planning and developing reviews.
Typology of Literature Reviews (adapted from Paré et al., 2015).
As shown in Table 9.1 , each review type addresses different kinds of research questions or objectives, which subsequently define and dictate the methods and approaches that need to be used to achieve the overarching goal(s) of the review. For example, in the case of narrative reviews, there is greater flexibility in searching and synthesizing articles ( Green et al., 2006 ). Researchers are often relatively free to use a diversity of approaches to search, identify, and select relevant scientific articles, describe their operational characteristics, present how the individual studies fit together, and formulate conclusions. On the other hand, systematic reviews are characterized by their high level of systematicity, rigour, and use of explicit methods, based on an “a priori” review plan that aims to minimize bias in the analysis and synthesis process (Higgins & Green, 2008). Some reviews are exploratory in nature (e.g., scoping/mapping reviews), whereas others may be conducted to discover patterns (e.g., descriptive reviews) or involve a synthesis approach that may include the critical analysis of prior research ( Paré et al., 2015 ). Hence, in order to select the most appropriate type of review, it is critical to know before embarking on a review project, why the research synthesis is conducted and what type of methods are best aligned with the pursued goals.
9.5. Concluding Remarks
In light of the increased use of evidence-based practice and research generating stronger evidence ( Grady et al., 2011 ; Lyden et al., 2013 ), review articles have become essential tools for summarizing, synthesizing, integrating or critically appraising prior knowledge in the eHealth field. As mentioned earlier, when rigorously conducted review articles represent powerful information sources for eHealth scholars and practitioners looking for state-of-the-art evidence. The typology of literature reviews we used herein will allow eHealth researchers, graduate students and practitioners to gain a better understanding of the similarities and differences between review types.
We must stress that this classification scheme does not privilege any specific type of review as being of higher quality than another ( Paré et al., 2015 ). As explained above, each type of review has its own strengths and limitations. Having said that, we realize that the methodological rigour of any review — be it qualitative, quantitative or mixed — is a critical aspect that should be considered seriously by prospective authors. In the present context, the notion of rigour refers to the reliability and validity of the review process described in section 9.2. For one thing, reliability is related to the reproducibility of the review process and steps, which is facilitated by a comprehensive documentation of the literature search process, extraction, coding and analysis performed in the review. Whether the search is comprehensive or not, whether it involves a methodical approach for data extraction and synthesis or not, it is important that the review documents in an explicit and transparent manner the steps and approach that were used in the process of its development. Next, validity characterizes the degree to which the review process was conducted appropriately. It goes beyond documentation and reflects decisions related to the selection of the sources, the search terms used, the period of time covered, the articles selected in the search, and the application of backward and forward searches ( vom Brocke et al., 2009 ). In short, the rigour of any review article is reflected by the explicitness of its methods (i.e., transparency) and the soundness of the approach used. We refer those interested in the concepts of rigour and quality to the work of Templier and Paré (2015) which offers a detailed set of methodological guidelines for conducting and evaluating various types of review articles.
To conclude, our main objective in this chapter was to demystify the various types of literature reviews that are central to the continuous development of the eHealth field. It is our hope that our descriptive account will serve as a valuable source for those conducting, evaluating or using reviews in this important and growing domain.
- Ammenwerth E., de Keizer N. An inventory of evaluation studies of information technology in health care. Trends in evaluation research, 1982-2002. International Journal of Medical Informatics. 2004; 44 (1):44–56. [ PubMed : 15778794 ]
- Anderson S., Allen P., Peckham S., Goodwin N. Asking the right questions: scoping studies in the commissioning of research on the organisation and delivery of health services. Health Research Policy and Systems. 2008; 6 (7):1–12. [ PMC free article : PMC2500008 ] [ PubMed : 18613961 ] [ CrossRef ]
- Archer N., Fevrier-Thomas U., Lokker C., McKibbon K. A., Straus S.E. Personal health records: a scoping review. Journal of American Medical Informatics Association. 2011; 18 (4):515–522. [ PMC free article : PMC3128401 ] [ PubMed : 21672914 ]
- Arksey H., O’Malley L. Scoping studies: towards a methodological framework. International Journal of Social Research Methodology. 2005; 8 (1):19–32.
- A systematic, tool-supported method for conducting literature reviews in information systems. Paper presented at the Proceedings of the 19th European Conference on Information Systems ( ecis 2011); June 9 to 11; Helsinki, Finland. 2011.
- Baumeister R. F., Leary M.R. Writing narrative literature reviews. Review of General Psychology. 1997; 1 (3):311–320.
- Becker L. A., Oxman A.D. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Overviews of reviews; pp. 607–631.
- Borenstein M., Hedges L., Higgins J., Rothstein H. Introduction to meta-analysis. Hoboken, nj : John Wiley & Sons Inc; 2009.
- Cook D. J., Mulrow C. D., Haynes B. Systematic reviews: Synthesis of best evidence for clinical decisions. Annals of Internal Medicine. 1997; 126 (5):376–380. [ PubMed : 9054282 ]
- Cooper H., Hedges L.V. In: The handbook of research synthesis and meta-analysis. 2nd ed. Cooper H., Hedges L. V., Valentine J. C., editors. New York: Russell Sage Foundation; 2009. Research synthesis as a scientific process; pp. 3–17.
- Cooper H. M. Organizing knowledge syntheses: A taxonomy of literature reviews. Knowledge in Society. 1988; 1 (1):104–126.
- Cronin P., Ryan F., Coughlan M. Undertaking a literature review: a step-by-step approach. British Journal of Nursing. 2008; 17 (1):38–43. [ PubMed : 18399395 ]
- Darlow S., Wen K.Y. Development testing of mobile health interventions for cancer patient self-management: A review. Health Informatics Journal. 2015 (online before print). [ PubMed : 25916831 ] [ CrossRef ]
- Daudt H. M., van Mossel C., Scott S.J. Enhancing the scoping study methodology: a large, inter-professional team’s experience with Arksey and O’Malley’s framework. bmc Medical Research Methodology. 2013; 13 :48. [ PMC free article : PMC3614526 ] [ PubMed : 23522333 ] [ CrossRef ]
- Davies P. The relevance of systematic reviews to educational policy and practice. Oxford Review of Education. 2000; 26 (3-4):365–378.
- Deeks J. J., Higgins J. P. T., Altman D.G. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Analysing data and undertaking meta-analyses; pp. 243–296.
- Deshazo J. P., Lavallie D. L., Wolf F.M. Publication trends in the medical informatics literature: 20 years of “Medical Informatics” in mesh . bmc Medical Informatics and Decision Making. 2009; 9 :7. [ PMC free article : PMC2652453 ] [ PubMed : 19159472 ] [ CrossRef ]
- Dixon-Woods M., Agarwal S., Jones D., Young B., Sutton A. Synthesising qualitative and quantitative evidence: a review of possible methods. Journal of Health Services Research and Policy. 2005; 10 (1):45–53. [ PubMed : 15667704 ]
- Finfgeld-Connett D., Johnson E.D. Literature search strategies for conducting knowledge-building and theory-generating qualitative systematic reviews. Journal of Advanced Nursing. 2013; 69 (1):194–204. [ PMC free article : PMC3424349 ] [ PubMed : 22591030 ]
- Grady B., Myers K. M., Nelson E. L., Belz N., Bennett L., Carnahan L. … Guidelines Working Group. Evidence-based practice for telemental health. Telemedicine Journal and E Health. 2011; 17 (2):131–148. [ PubMed : 21385026 ]
- Green B. N., Johnson C. D., Adams A. Writing narrative literature reviews for peer-reviewed journals: secrets of the trade. Journal of Chiropractic Medicine. 2006; 5 (3):101–117. [ PMC free article : PMC2647067 ] [ PubMed : 19674681 ]
- Greenhalgh T., Wong G., Westhorp G., Pawson R. Protocol–realist and meta-narrative evidence synthesis: evolving standards ( rameses ). bmc Medical Research Methodology. 2011; 11 :115. [ PMC free article : PMC3173389 ] [ PubMed : 21843376 ]
- Gurol-Urganci I., de Jongh T., Vodopivec-Jamsek V., Atun R., Car J. Mobile phone messaging reminders for attendance at healthcare appointments. Cochrane Database System Review. 2013; 12 cd 007458. [ PMC free article : PMC6485985 ] [ PubMed : 24310741 ] [ CrossRef ]
- Hart C. Doing a literature review: Releasing the social science research imagination. London: SAGE Publications; 1998.
- Higgins J. P. T., Green S., editors. Cochrane handbook for systematic reviews of interventions: Cochrane book series. Hoboken, nj : Wiley-Blackwell; 2008.
- Jesson J., Matheson L., Lacey F.M. Doing your literature review: traditional and systematic techniques. Los Angeles & London: SAGE Publications; 2011.
- King W. R., He J. Understanding the role and methods of meta-analysis in IS research. Communications of the Association for Information Systems. 2005; 16 :1.
- Kirkevold M. Integrative nursing research — an important strategy to further the development of nursing science and nursing practice. Journal of Advanced Nursing. 1997; 25 (5):977–984. [ PubMed : 9147203 ]
- Kitchenham B., Charters S. ebse Technical Report Version 2.3. Keele & Durham. uk : Keele University & University of Durham; 2007. Guidelines for performing systematic literature reviews in software engineering.
- Kitsiou S., Paré G., Jaana M. Systematic reviews and meta-analyses of home telemonitoring interventions for patients with chronic diseases: a critical assessment of their methodological quality. Journal of Medical Internet Research. 2013; 15 (7):e150. [ PMC free article : PMC3785977 ] [ PubMed : 23880072 ]
- Kitsiou S., Paré G., Jaana M. Effects of home telemonitoring interventions on patients with chronic heart failure: an overview of systematic reviews. Journal of Medical Internet Research. 2015; 17 (3):e63. [ PMC free article : PMC4376138 ] [ PubMed : 25768664 ]
- Levac D., Colquhoun H., O’Brien K. K. Scoping studies: advancing the methodology. Implementation Science. 2010; 5 (1):69. [ PMC free article : PMC2954944 ] [ PubMed : 20854677 ]
- Levy Y., Ellis T.J. A systems approach to conduct an effective literature review in support of information systems research. Informing Science. 2006; 9 :181–211.
- Liberati A., Altman D. G., Tetzlaff J., Mulrow C., Gøtzsche P. C., Ioannidis J. P. A. et al. Moher D. The prisma statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. Annals of Internal Medicine. 2009; 151 (4):W-65. [ PubMed : 19622512 ]
- Lyden J. R., Zickmund S. L., Bhargava T. D., Bryce C. L., Conroy M. B., Fischer G. S. et al. McTigue K. M. Implementing health information technology in a patient-centered manner: Patient experiences with an online evidence-based lifestyle intervention. Journal for Healthcare Quality. 2013; 35 (5):47–57. [ PubMed : 24004039 ]
- Mickan S., Atherton H., Roberts N. W., Heneghan C., Tilson J.K. Use of handheld computers in clinical practice: a systematic review. bmc Medical Informatics and Decision Making. 2014; 14 :56. [ PMC free article : PMC4099138 ] [ PubMed : 24998515 ]
- Moher D. The problem of duplicate systematic reviews. British Medical Journal. 2013; 347 (5040) [ PubMed : 23945367 ] [ CrossRef ]
- Montori V. M., Wilczynski N. L., Morgan D., Haynes R. B., Hedges T. Systematic reviews: a cross-sectional study of location and citation counts. bmc Medicine. 2003; 1 :2. [ PMC free article : PMC281591 ] [ PubMed : 14633274 ]
- Mulrow C. D. The medical review article: state of the science. Annals of Internal Medicine. 1987; 106 (3):485–488. [ PubMed : 3813259 ] [ CrossRef ]
- Evidence-based information systems: A decade later. Proceedings of the European Conference on Information Systems ; 2011. Retrieved from http://aisel .aisnet.org/cgi/viewcontent .cgi?article =1221&context =ecis2011 .
- Okoli C., Schabram K. A guide to conducting a systematic literature review of information systems research. ssrn Electronic Journal. 2010
- Otte-Trojel T., de Bont A., Rundall T. G., van de Klundert J. How outcomes are achieved through patient portals: a realist review. Journal of American Medical Informatics Association. 2014; 21 (4):751–757. [ PMC free article : PMC4078283 ] [ PubMed : 24503882 ]
- Paré G., Trudel M.-C., Jaana M., Kitsiou S. Synthesizing information systems knowledge: A typology of literature reviews. Information & Management. 2015; 52 (2):183–199.
- Patsopoulos N. A., Analatos A. A., Ioannidis J.P. A. Relative citation impact of various study designs in the health sciences. Journal of the American Medical Association. 2005; 293 (19):2362–2366. [ PubMed : 15900006 ]
- Paul M. M., Greene C. M., Newton-Dame R., Thorpe L. E., Perlman S. E., McVeigh K. H., Gourevitch M.N. The state of population health surveillance using electronic health records: A narrative review. Population Health Management. 2015; 18 (3):209–216. [ PubMed : 25608033 ]
- Pawson R. Evidence-based policy: a realist perspective. London: SAGE Publications; 2006.
- Pawson R., Greenhalgh T., Harvey G., Walshe K. Realist review—a new method of systematic review designed for complex policy interventions. Journal of Health Services Research & Policy. 2005; 10 (Suppl 1):21–34. [ PubMed : 16053581 ]
- Petersen K., Vakkalanka S., Kuzniarz L. Guidelines for conducting systematic mapping studies in software engineering: An update. Information and Software Technology. 2015; 64 :1–18.
- Petticrew M., Roberts H. Systematic reviews in the social sciences: A practical guide. Malden, ma : Blackwell Publishing Co; 2006.
- Rousseau D. M., Manning J., Denyer D. Evidence in management and organizational science: Assembling the field’s full weight of scientific knowledge through syntheses. The Academy of Management Annals. 2008; 2 (1):475–515.
- Rowe F. What literature review is not: diversity, boundaries and recommendations. European Journal of Information Systems. 2014; 23 (3):241–255.
- Shea B. J., Hamel C., Wells G. A., Bouter L. M., Kristjansson E., Grimshaw J. et al. Boers M. amstar is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. Journal of Clinical Epidemiology. 2009; 62 (10):1013–1020. [ PubMed : 19230606 ]
- Shepperd S., Lewin S., Straus S., Clarke M., Eccles M. P., Fitzpatrick R. et al. Sheikh A. Can we systematically review studies that evaluate complex interventions? PLoS Medicine. 2009; 6 (8):e1000086. [ PMC free article : PMC2717209 ] [ PubMed : 19668360 ]
- Silva B. M., Rodrigues J. J., de la Torre Díez I., López-Coronado M., Saleem K. Mobile-health: A review of current state in 2015. Journal of Biomedical Informatics. 2015; 56 :265–272. [ PubMed : 26071682 ]
- Smith V., Devane D., Begley C., Clarke M. Methodology in conducting a systematic review of systematic reviews of healthcare interventions. bmc Medical Research Methodology. 2011; 11 (1):15. [ PMC free article : PMC3039637 ] [ PubMed : 21291558 ]
- Sylvester A., Tate M., Johnstone D. Beyond synthesis: re-presenting heterogeneous research literature. Behaviour & Information Technology. 2013; 32 (12):1199–1215.
- Templier M., Paré G. A framework for guiding and evaluating literature reviews. Communications of the Association for Information Systems. 2015; 37 (6):112–137.
- Thomas J., Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. bmc Medical Research Methodology. 2008; 8 (1):45. [ PMC free article : PMC2478656 ] [ PubMed : 18616818 ]
- Reconstructing the giant: on the importance of rigour in documenting the literature search process. Paper presented at the Proceedings of the 17th European Conference on Information Systems ( ecis 2009); Verona, Italy. 2009.
- Webster J., Watson R.T. Analyzing the past to prepare for the future: Writing a literature review. Management Information Systems Quarterly. 2002; 26 (2):11.
- Whitlock E. P., Lin J. S., Chou R., Shekelle P., Robinson K.A. Using existing systematic reviews in complex systematic reviews. Annals of Internal Medicine. 2008; 148 (10):776–782. [ PubMed : 18490690 ]
This publication is licensed under a Creative Commons License, Attribution-Noncommercial 4.0 International License (CC BY-NC 4.0): see https://creativecommons.org/licenses/by-nc/4.0/
- Cite this Page Paré G, Kitsiou S. Chapter 9 Methods for Literature Reviews. In: Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.
- PDF version of this title (4.5M)
- Disable Glossary Links
In this Page
- Overview of the Literature Review Process and Steps
- Types of Review Articles and Brief Illustrations
- Concluding Remarks
- PMC PubMed Central citations
- PubMed Links to PubMed
- Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Ev... Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Evidence-based Approach
Your browsing activity is empty.
Activity recording is turned off.
Turn recording back on
Connect with NLM
National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894
Web Policies FOIA HHS Vulnerability Disclosure
Help Accessibility Careers
Research Process :: Step by Step
- Select Topic
- Identify Keywords
- Background Information
- Develop Research Questions
- Refine Topic
- Search Strategy
- Popular Databases
- Evaluate Sources
- Types of Periodicals
- Reading Scholarly Articles
- Primary & Secondary Sources
- Organize / Take Notes
- Writing & Grammar Resources
- Annotated Bibliography
- Literature Review
- Citation Styles
- Privacy / Confidentiality
- Research Process
- Selecting Your Topic
- Identifying Keywords
- Gathering Background Info
- Evaluating Sources
Organize the literature review into sections that present themes or identify trends, including relevant theory. You are not trying to list all the material published, but to synthesize and evaluate it according to the guiding concept of your thesis or research question.
What is a literature review?
A literature review is an account of what has been published on a topic by accredited scholars and researchers. Occasionally you will be asked to write one as a separate assignment, but more often it is part of the introduction to an essay, research report, or thesis. In writing the literature review, your purpose is to convey to your reader what knowledge and ideas have been established on a topic, and what their strengths and weaknesses are. As a piece of writing, the literature review must be defined by a guiding concept (e.g., your research objective, the problem or issue you are discussing, or your argumentative thesis). It is not just a descriptive list of the material available, or a set of summaries
A literature review must do these things:
- be organized around and related directly to the thesis or research question you are developing
- synthesize results into a summary of what is and is not known
- identify areas of controversy in the literature
- formulate questions that need further research
Ask yourself questions like these:
- What is the specific thesis, problem, or research question that my literature review helps to define?
- What type of literature review am I conducting? Am I looking at issues of theory? methodology? policy? quantitative research (e.g. on the effectiveness of a new procedure)? qualitative research (e.g., studies of loneliness among migrant workers)?
- What is the scope of my literature review? What types of publications am I using (e.g., journals, books, government documents, popular media)? What discipline am I working in (e.g., nursing psychology, sociology, medicine)?
- How good was my information seeking? Has my search been wide enough to ensure I've found all the relevant material? Has it been narrow enough to exclude irrelevant material? Is the number of sources I've used appropriate for the length of my paper?
- Have I critically analyzed the literature I use? Do I follow through a set of concepts and questions, comparing items to each other in the ways they deal with them? Instead of just listing and summarizing items, do I assess them, discussing strengths and weaknesses?
- Have I cited and discussed studies contrary to my perspective?
- Will the reader find my literature review relevant, appropriate, and useful?
Ask yourself questions like these about each book or article you include:
- Has the author formulated a problem/issue?
- Is it clearly defined? Is its significance (scope, severity, relevance) clearly established?
- Could the problem have been approached more effectively from another perspective?
- What is the author's research orientation (e.g., interpretive, critical science, combination)?
- What is the author's theoretical framework (e.g., psychological, developmental, feminist)?
- What is the relationship between the theoretical and research perspectives?
- Has the author evaluated the literature relevant to the problem/issue? Does the author include literature taking positions she or he does not agree with?
- In a research study, how good are the basic components of the study design (e.g., population, intervention, outcome)? How accurate and valid are the measurements? Is the analysis of the data accurate and relevant to the research question? Are the conclusions validly based upon the data and analysis?
- In material written for a popular readership, does the author use appeals to emotion, one-sided examples, or rhetorically-charged language and tone? Is there an objective basis to the reasoning, or is the author merely "proving" what he or she already believes?
- How does the author structure the argument? Can you "deconstruct" the flow of the argument to see whether or where it breaks down logically (e.g., in establishing cause-effect relationships)?
- In what ways does this book or article contribute to our understanding of the problem under study, and in what ways is it useful for practice? What are the strengths and limitations?
- How does this book or article relate to the specific thesis or question I am developing?
Text written by Dena Taylor, Health Sciences Writing Centre, University of Toronto
- << Previous: Annotated Bibliography
- Next: Step 5: Cite Sources >>
- Last Updated: Nov 7, 2023 1:00 PM
- URL: https://libguides.uta.edu/researchprocess
University of Texas Arlington Libraries 702 Planetarium Place · Arlington, TX 76019 · 817-272-3000
- Internet Privacy
- Problems with a guide? Contact Us.
- Schools & departments
A general guide on how to conduct and write a literature review.
Please check course or programme information and materials provided by teaching staff , including your project supervisor, for subject-specific guidance.
What is a literature review?
A literature review is a piece of academic writing demonstrating knowledge and understanding of the academic literature on a specific topic placed in context. A literature review also includes a critical evaluation of the material; this is why it is called a literature review rather than a literature report. It is a process of reviewing the literature, as well as a form of writing.
To illustrate the difference between reporting and reviewing, think about television or film review articles. These articles include content such as a brief synopsis or the key points of the film or programme plus the critic’s own evaluation. Similarly the two main objectives of a literature review are firstly the content covering existing research, theories and evidence, and secondly your own critical evaluation and discussion of this content.
Usually a literature review forms a section or part of a dissertation, research project or long essay. However, it can also be set and assessed as a standalone piece of work.
What is the purpose of a literature review?
…your task is to build an argument, not a library. Rudestam, K.E. and Newton, R.R. (1992) Surviving your dissertation: A comprehensive guide to content and process. California: Sage, p49.
In a larger piece of written work, such as a dissertation or project, a literature review is usually one of the first tasks carried out after deciding on a topic. Reading combined with critical analysis can help to refine a topic and frame research questions. Conducting a literature review establishes your familiarity with and understanding of current research in a particular field before carrying out a new investigation. After doing a literature review, you should know what research has already been done and be able to identify what is unknown within your topic.
When doing and writing a literature review, it is good practice to:
- summarise and analyse previous research and theories;
- identify areas of controversy and contested claims;
- highlight any gaps that may exist in research to date.
Conducting a literature review
Focusing on different aspects of your literature review can be useful to help plan, develop, refine and write it. You can use and adapt the prompt questions in our worksheet below at different points in the process of researching and writing your review. These are suggestions to get you thinking and writing.
Developing and refining your literature review (pdf)
Developing and refining your literature review (Word)
Developing and refining your literature review (Word rtf)
Writing a literature review has a lot in common with other assignment tasks. There is advice on our other pages about thinking critically, reading strategies and academic writing. Our literature review top tips suggest some specific things you can do to help you submit a successful review.
Literature review top tips (pdf)
Literature review top tips (Word rtf)
Our reading page includes strategies and advice on using books and articles and a notes record sheet grid you can use.
Reading at university
The Academic writing page suggests ways to organise and structure information from a range of sources and how you can develop your argument as you read and write.
The Critical thinking page has advice on how to be a more critical researcher and a form you can use to help you think and break down the stages of developing your argument.
As with other forms of academic writing, your literature review needs to demonstrate good academic practice by following the Code of Student Conduct and acknowledging the work of others through citing and referencing your sources.
Good academic practice
As with any writing task, you will need to review, edit and rewrite sections of your literature review. The Editing and proofreading page includes tips on how to do this and strategies for standing back and thinking about your structure and checking the flow of your argument.
Editing and proofreading
Guidance on literature searching from the University Library
The Academic Support Librarians have developed LibSmart I and II, Learn courses to help you develop and enhance your digital research skills and capabilities; from getting started with the Library to managing data for your dissertation.
Searching using the library’s DiscoverEd tool: DiscoverEd
Finding resources in your subject: Subject guides
The Academic Support Librarians also provide one-to-one appointments to help you develop your research strategies.
1 to 1 support for literature searching and systematic reviews
Advice to help you optimise use of Google Scholar, Google Books and Google for your research and study: Using Google
Managing and curating your references
A referencing management tool can help you to collect and organise and your source material to produce a bibliography or reference list.
Referencing and reference management
Information Services provide access to Cite them right online which is a guide to the main referencing systems and tells you how to reference just about any source (EASE log-in may be required).
Cite them right
Published study guides
There are a number of scholarship skills books and guides available which can help with writing a literature review. Our Resource List of study skills guides includes sections on Referencing, Dissertation and project writing and Literature reviews.
Study skills guides