Systematic Reviews and Evidence Syntheses: Searching for studies

In planning your review you will have broken down your topic using a question formulation tool, e.g. PICO or PCC. In preparing your search strategy you will look again at the PICO and select which key concepts will be included in the search, and which will be used as inclusion / exclusion criteria at the title / abstract or full-text screening stage. For a sensitive search, it’s common practice to select two (sometimes three) elements of the PICO for searching, often the Population and Intervention elements. For example, for this PICO: 

How do delayed antibiotic prescriptions for respiratory infections affect patient and service outcomes compared to immediate / no prescription? 

Population = Patients with respiratory infections 

Intervention = Delayed antibiotic prescription  

Comparison = Immediate or no prescription 

Outcomes = time to recovery, repeat GP appointment, emergency hospitalisation, patient satisfaction... 

The key elements for starting your search would be respiratory infections, antibiotics and delayed prescribing. For this question, we have multiple comparators and multiple outcomes, if we added keywords for these concepts, we might overlook significant synonyms and miss relevant papers. By focusing on the population and intervention we will automatically retrieve papers reporting any or no comparator and any outcomes, primary or secondary. 

In your scoping searches you will have identified related reviews and a small pool of key studies. These can be a helpful source of title / abstract keywords and subject headings for your search strategy. As a first step you may want to skim the titles and abstracts of relevant papers and categorise terminology and synonyms used by authors writing about your topic, e.g. 

• Respiratory infections, respiratory tract infections, RTI, common cold, influenza... 

• Antibiotics, anti-biotics, antibacterials, beta lactams, penicillin... 

• Delayed prescribing, delayed prescription, delaying prescriptions... 

Alongside variations in terminology you need to be aware of differences in spelling, acronyms, broader concepts (respiratory infections) and narrower concepts (common cold), drug classes and individual drug names.  

Keywords, also called free-text terms, are only one part of your search. Many bibliographic databases tag articles using subject headings, also called index terms or thesaurus terms. These are individual to each database and will need to be incorporated into your search strategy alongside the author keywords. 

To see example search terms for our question above, have a look at the search methods reported in the Cochrane review on Immediate versus delayed versus no antibiotics for respiratory infections. If you do use someone else’s search strategy as inspiration this should be fully acknowledged and cited in your review. 

If you’re a frequent user of MEDLINE / PubMed there are several helpful tools that can be used to perform an analysis on a group of relevant studies to identify frequently occurring keywords and / or subject headings, these include: 

Text mining tools are increasingly being used to aid the development of search strategies, but it isn’t a fully automated process. As a searcher you will need to use your discretion when you use these tools, bearing in mind that they don’t behave consistently across different topics. Further advice is forthcoming from information specialists revising Chapter 4: Searching and Selecting Studies in the Cochrane Handbook. In the meantime, CADTH’s Text Mining Opportunities: White Paper provides useful guidance. 

In some cases we breakdown our question into PICOS, with the S identifying the Study Type that will provide the best evidence to answer the question. For example, if we’re conducting a review looking at the effectiveness of an intervention, we might be guided by Levels of Evidence to restrict our search to randomised controlled trials. The appropriate study type will differ from question to question, and in some cases (scoping reviews, realist reviews...) it may not be advisable to restrict by study type. 

In conducting a sensitive search, limiting the search by using in-built publication type filters is likely to miss relevant studies. There is now a bank of tried, tested and validated search filters available from the InterTASC ISSG search filter resource and these can be applied to searches across different databases. 

The development of a search strategy is an iterative process and is initially conducted on one database, usually MEDLINE / PubMed or Embase. You will need to test your free-text search terms, subject headings and their combination (AND / OR) to get a sensitive search, without retrieving too many irrelevant papers. For a comprehensive search, it is inevitable that irrelevant papers will be included, and these will need to be excluded during the screening process. 

To learn more about building a search strategy you can register for our iSkills for Medicine: Searching Systematically workshop or view our literature searching videos. 

To make sure there are no omissions, mistakes or unknown biases within your search strategy, it is recommended that the search be independently assessed. The PRESS Peer Review of Electronic Search Strategies can be used to review the search in a structured manner. Carrie Price in her PRESS Five minute Friday video gives a quick introduction to the process. 

Once the search strategy has been developed for one database it will need to be adapted across all relevant bibliographic databases. The MECIR manual providing guidance on the conduct of Cochrane Intervention Reviews specifies MEDLINE, Embase and CENTRAL as compulsory databases. Although specific for intervention reviews, three databases as a minimum are a good starting point for most reviews.  

These are some of the databases relevant to systematic reviews in health care. Additional or alternative databases should be searched depending on your question. 

• CINAHL (Cumulative Index to Nursing and Allied Health Literature) 

• Cochrane Library, including Cochrane Database of Systematic Reviews (CDSR) and Cochrane Central Register of Controlled Trials (CENTRAL) 

• EMBASE: Excerpta Medica 

• MEDLINE  

• PsycINFO 

For staff and students at the University of Oxford these can be accessed via Databases A-Z with an Oxford SSO. For NHS staff, including those at the OUH NHS Foundation Trust, these can be accessed through the NHS Knowledge and Library Hub with an NHS OpenAthens account. 

Ongoing studies 

It is important to be aware of ongoing studies as this could affect the decision on when to publish and/or update the review. You should also report details of ongoing studies in your review.  

Trial registrations can be identified through a search of CENTRAL via Cochrane Library. However, it is advisable to search two additional trial registries if you’re conducting a review of intervention studies: 

Reports of studies are also increasingly reported prior to peer review and publication as journal articles. You may wish to conduct a search of individual preprint archives (medRxiv) or search a platform which conducts a federated search across preprint archives (Europe PMC) 

As mentioned already, a sensitive search is essential for an evidence review, and while a search of multiple bibliographic databases and trial registers is the key part of this process, supplementary searching is essential to verify that no studies have been missed. 

Citation tracking 

This process will be conducted once the initial results have been screened and you have a list of included studies. You can screen the reference lists of included studies (backward citation tracking) and/or search for papers that have cited your included studies (forward citation tracking). GoogleScholar, PubMed, Scopus and Web of Science Core Collection are useful databases for this process, you can also automate the search with Citation Chaser. See our video on Citation Searching for a quick demonstration. 

Grey literature 

Grey literature is generally considered to be material published or made available by organisations or individuals, not through commercial publishers. This can include:  

• Conference abstracts not published in journal supplements 

• Theses 

• Preprints (discussed under ongoing studies above) 

• Government documents and organisational reports 

• Clinical study reports and regulatory documents 

• Personal communication, social media and blog posts 

 
Searching for grey literature can be problematic, as it isn't collected, organised or stored in a consistent way. These issues are explored by Dickersin, Stansfield and Bangpan. You will need to be flexible in your approach, depending on the type of grey literature you need – this will differ from review to review. 

CADTH’s publication Grey Matters: a practical tool for searching health-related grey literature gives guidance on identifying organisational and HTA reports 

Look at the Databases A-Z for resources for finding theses (Proquest Dissertations & Theses (Global), eTHOS), conference abstracts (Embase, Web of Science) and organisational reports (Overton) 

Searching specific organisational websites and a subject search on a web search engine can also be useful approaches. Look at our video on web searching for grey literature for top tips. 

You will need to document your search in order to comply with PRISMA-S – this will be covered in more detail in a forthcoming section on Writing Up. 

At this stage, be sure to keep a record of: 

• Databases searched – including version and date searched 

• Full electronic strategy for each databases – these must be be downloaded from the database at time of search 

• Any limits applied, e.g. language, date, study type 

• Other search methods applied, e.g. screening reference lists, forward citation searching and contact with authors 

• Details of any web searches including name of website, URL, keywords used and date searched.