A messy, time-consuming treasure hunt
Grey literature is messier and more time-consuming to find than academic journal articles. But for many health-related topics (particularly on epidemiology, health service organisation and delivery, and qualitative aspects rather than clinical effectiveness), a review is incomplete without it.
The nature of grey literature
In their seminal paper A difficult business: finding the evidence for social science reviews, Lesley Grayson and Alan Gomersall discussed the importance of grey literature for social policy and related fields, including capturing what is already known. Although they were writing in 2003, many of the challenges with grey lit use persist: broadly being that grey lit is more difficult to find and organise for an evidence review.
Types and sources of grey literature
For health-related topics, the types of grey literature of most interest are:
Conference papers – abstracts, presentations – available in medical literature databases. Some studies will eventually be published as journal articles.
Dissertations and theses – again, some researchers are likely to publish their findings as journal articles.
Reports – from government agencies, not-for-profit organisations, academic institutions, for-profit companies (e.g. consultancy white papers) – probably the hardest to find
- May require searching key organisations’ websites
- Other options are to search the British Library catalogue, WorldCat, Google Advanced/Scholar or subscription sources such as Scopus.
Some guidelines, although many are published as journal articles
A different approach to finding grey literature is interviewing key opinion leaders/experts in the field.
Which types of review need to draw on grey literature?
Apart from topics which do not relate directly to clinical effectiveness, the following examples from my recent projects call for grey literature.
- Topics inherently focussed on very recent outputs, e.g. Covid-19 vaccination
- Commercially-applied topics, for example looking at economic aspects of new-to-market treatment
- Concepts that are not well covered by MeSH thesaurus terms, e.g. the concept of cure in oncology
Organising grey literature
Some reports and web sources can be imported to reference management software, but success and completeness depends on metadata (variable) quality. A likely outcome is partial import that will then need cleaning up of record fields, e.g. titles often import incorrectly (allow time to do this task).
Reporting the search
Again, this task can be time-consuming and hard to systematise. A recent extension to the PRISMA statement offers pragmatic advice on how to manage reporting web searches that selectively show algorithm-based results rather than a reproducible volume.
Grayson L, Gomersall A. A difficult business: finding the evidence for social science reviews. London: ESRC UK Centre for Evidence Based Policy and Practice, Queen Mary University of London. 2003.
Paez A. Gray literature: An important resource in systematic reviews. J Evid Based Med. 2017 Aug;10(3):233-40.
Rethlefsen ML, Kirtley S, Waffenschmidt S. et al. PRISMA-S: an extension to the PRISMA Statement for Reporting Literature Searches in Systematic Reviews. Syst Rev. 2021 Jan;10(39). https://doi.org/10.1186/s13643-020-01542-z.