One of the things that faculty often complain about is that students don’t adequately track down and cite enough relevant material for their term papers and projects. This problem isn’t confined to undergraduates. A study in the January 4, 2011 issue of the Annals of Internal Medicine by Karen Robinson and Steven Goodman finds that medical researchers aren’t doing a very good job of citing previous research either.
Specifically, Robinson and Goodman looked at reports of randomized, controlled trials to determine if the authors cited previous, related trials. Citing previous trials is an important part of putting the results of the current trial in context, and in the case of medicine, may help save lives.
In order to do this study, the authors used meta-analysis to locate groups of related papers. They reasoned that if the studies were similar enough to group mathematically, they were similar enough to cite each other. The allowed for a 1-year gap between an original publication and a citation.
Overall, they found that only 25% of relevant papers were actually cited.
Why might a citation not be included? I can think of a few reasons.
- The authors couldn’t find the previous study
- The authors found the previous study but didn’t think it was relevant enough to cite
- The authors found the study and purposefully excluded it for some nefarious purpose
Robinson and Goodman seem to favor the first explanation most of all:
The obvious remedy – requiring a systematic review of relevant literature [before an RCT is funded] – is hampered by a lack of necessary skills and resources.
This obviously speaks to the importance of information literacy skills in both undergraduates and medical school students. One of the most troubling things about the article results was Robinson and Goodman’s determination that a very simple PubMed search could locate most of the articles on one of the topics assessed.
An interesting recommendation that Robinson and Goodman repeat throughout the article is to suggest that a description of the search strategy for prior results be included in the final published article (and they follow their own advice in an appendix to the article).
Of course, it is hard to believe that this problem is limited to just the authors of randomized control trials in biomedicine. It wouldn’t take much to convince me that this problem exists throughout scholarly work, restricting the speed at which new discoveries are made. I would bet that the problem can get particularly difficult in interdisciplinary areas.
We need to start with our undergraduates and convince them that it isn’t enough just to find the minimum number of required sources, but to really get at the heart of previous work on a topic. This leads naturally into the topic of getting students to pick manageable project topics. Of course, undergraduates like clear guidelines (and for the most part this is good teaching strategy), but upper level undergraduates should be able to handle the requirement that they find most of the relevant literature on a topic.
Robinson KA, & Goodman SN (2011). A systematic examination of the citation of prior research in reports of randomized, controlled trials. Annals of internal medicine, 154 (1), 50-5 PMID: 21200038
- Trial in a Vacuum: Study of Studies Shows Few Citations from the New York Times
- Citation-amnesia paper published from ScienceNews