For many researchers, the citation is a make-or-break concept. Most ranking algorithms use citations to determine a journal’s influence or impact. Publication in “high impact” journals is often the key to tenure and promotion, and the number of times an article has been cited is often widely touted in tenure and promotion packets.
With careers, funding and much else riding on citation, it would be useful for scholars and librarians to know why a particular item gets cited. We’d all like to think that the only reason an article is cited is because it’s content is relevant (and more relevant than other items) to the study at hand.
Unfortunately, there is some evidence to suggest that other, non-content, factors influence the likelihood of an item being cited.
Big caveat: The quality of these studies is highly variable and their results are sometimes contradictory. Correlation does not equal causation.
Nevertheless, most of the non-content factors influencing citation rate relate to article discoverability. You can’t be cited if you can’t be read, and you can’t be read if you can’t be found. How likely is an article to be found in a database? Was the article discussed in a newspaper or other popular science forum? Does the title clearly explain what the article is about (and make you want to read more)? Is the article already connected to a wide circle of readers via multiple authors or large universities? While there are classic examples of important scientific publications published in obscure journals, those are the exception and not the norm.
So, in no particular order, here are a few things that folks suggest might influence how often your article is cited:
A lot of research has looked into various aspects of article titles on subsequent citations.
- Type of title – In an interesting study looking at article titles from PLoS journals, Jamali and Nikzad (2011) wondered if the type of article title affected the citation rate of an article. In general, they found that article titles that asked a question were downloaded more but cited less than descriptive or declarative titles. Interestingly, Ball (2009) found that the number of such interrogative titles have increased 50% – 200% in the last 40 years.
- Length of title – Jamali and Nikzad suggest that articles with longer titles are downloaded and cited less, and Moore (2010) in a quick study found no correlation. However, Habibzadeh and Yadollahie (2010) suggested that longer titles are cited more (especially in high impact factor journals) and a positive correlation between article title length and citation rate was found by Jacques and Sabire (2009).
- Specific terms – Disciplinary abbreviations (very specific keywords) may lead to more citations (Moore, 2010), where as articles with specific country names in the title might be cited less (Jacques and Sabire, 2009).
- Humorous titles – To my disappointment, a study of articles with amusing titles in prestigious psychology journals by Sagi and Yechiam (2008) found that these articles were less likely to be cited than other articles with unfunny articles. Since funny titles are often less descriptive of the actual research, these articles could be more difficult to find in databases.
Positive Results – There is strong evidence to suggest that positive results are much more likely to be submitted and published than negative results. It seems as though positive results are also more likely to be cited. Banobi et al. (2011) found that rebuttal articles (either technical reports or full length articles) were less likely to be cited than the original articles, i.e. the articles with positive results were more likely to be cited. This correlates well with the results of Leimy and Koricheva (2005) who found that articles that successfully proved their original hypothesis were more likely to be cited than articles that disproved the original hypothesis
Number of authors – Leimu and Koricheva (2005) found a positive correlation between the number of authors and the number of citations in the ecological literature, while Kulkarni et al. (2007) found that group authorship in medical journals increased citation counts by 11.1. However, a blog post by Moore (2010) suggested that isn’t wasn’t the number of authors that were important, but their reputation. A recent study of the chemical literature that was able to account for article quality (as measured by reviewers rating) found a correlation with author reputation but no correlation to the number of authors (Bornmann et al. 2012).
Industry relationship – Studying medial journals, Kulkarni et al. (2007) found that industry funded research that reported results beneficial to the industry (i.e. a medical device that worked or a drug that didn’t show harmful side effects) was more likely to be cited than non-instustry funded, negative research.
Data sharing – Piwowar et al. (2007) found that within a specific scholarly community (cancer microarray clinical trial publications) free availability of research data let to a higher citation rate, independent of journal impact factor.
Open Access – Lots of studies have been done with mixed results. A slightly higher number of studies seem to suggest that open access leads to higher citations (See the excellent review article by Wagner (2010)).
Popular press coverage – It makes intuitive sense that journal articles spotlighted by the popular press might be cited more, but this is difficult to prove. Perhaps the press is merely good at identifying those articles that would be highly cited anyway. Phillips et. al (1991) were able to take advantage of an interesting situation when the New York Times went on strike in 1978 but continued to produce a “paper of record” that was never published. Phillips et. al. (1991) found that items written about in the “paper of record” but not published were no more likely to be cited than other articles.
Length of your bibliography – A 2009 study by Webster et al. (2009) suggests a correlation between the length of an articles bibliography and the number of times it is later cited. They suggest a “I’ll cite you since you cited me” mentality, but online commentators suggest that this is merely a specious relationship (See Corbyn, 2010, and comments therein).
So, if you want to publish a paper that gets the highest number of citations, what should you do? Do your study with a large number of prestigious co-authors. Submit your long article containing positive results and a big bibliography to a open access journal. Say something nice about a pharmaceutical company. Share your data and get the New York Times to write about it.
Oh, and it might be useful to have some interesting and solid science in there somewhere.
Really Long Bibliography:
Ball, R. (2009). Scholarly communication in transition: The use of question marks in the titles of scientific articles in medicine, life sciences and physics 1966–2005. Scientometrics, 79(3), 667–679. Retrieved from: http://www.akademiai.com/index/UH466Q5P3722N37L.pdf
Banobi, J. A., Branch, T. A., & Hilborn, R. (2011). Do rebuttals affect future science? Ecosphere, 2(3), art37. doi:10.1890/ES10-00142.1
Bornmann, L., Schier, H., Marx, W., & Daniel, H. D. (2012). What factors determine citation counts of publications in chemistry besides their quality? Journal of Informetrics, 6(1), 11-18. Elsevier Ltd. doi:10.1016/j.joi.2011.08.004
Corbyn, Z. (2010). An easy way to boost a paper’s citations. Nature. Nature Publishing Group. doi:10.1038/news.2010.406
Habibzadeh, F., & Yadollahie, M. (2010). Are Shorter Article Titles More Attractive for Citations? Cross-sectional Study of 22 Scientific Journals. Croatian Medical Journal, 51(2), 165-170. doi:10.3325/cmj.2010.51.165
Jacques, T. S., & Sebire, N. J. (2010). The impact of article titles on citation hits: an analysis of general and specialist medical journals. JRSM short reports, 1(1), 2. doi:10.1258/shorts.2009.100020
Jamali, H. R., & Nikzad, M. (2011). Article title type and its relation with the number of downloads and citations. Scientometrics, (49), 653-661. doi:10.1007/s11192-011-0412-z
Kulkarni, A. V., Busse, J. W., & Shams, I. (2007). Characteristics associated with citation rate of the medical literature. PloS one, 2(5), e403. doi:10.1371/journal.pone.0000403
Leimu, R., & Koricheva, J. (2005). What determines the citation frequency of ecological papers? Trends in ecology & evolution, 20(1), 28-32. doi:10.1016/j.tree.2004.10.010
Moore, A. (2010). Do Article Title Attributes Influence Citations? Wiley-Blackwell Publishing News. Retrieved December 16, 2011, from http://blogs.wiley.com/publishingnews/2010/09/02/do-article-title-attributes-influence-citations/
Phillips, D. P., Kanter, E. J., Bednarczyk, B., & Tastad, P. L. (1991). Importance of the Lay Press in the Transmission of Medical Knowledge to the Scientific Community. The New England Journal of Medicine, 325(16), 1180-1183. Available via: http://www.ncbi.nlm.nih.gov/pubmed/1891034
Piwowar, H. A., Day, R. S., & Fridsma, D. B. (2007). Sharing detailed research data is associated with increased citation rate. PloS ONE, 2(3), e308. doi:10.1371/journal.pone.0000308
Sagi, I., & Yechiam, E. (2008). Amusing titles in scientific journals and article citation. Journal of Information Science, 34(5), 680-687. doi:10.1177/0165551507086261
Wagner, A. B. (2010). Open access citation advantage: an annotated bibliography. Issues in Science and Technology Librarianship, (60). Retrieved from http://www.istl.org/10-winter/article2.html
Webster, G. D., Jonason, P. K., & Schember, T. O. (2009). Hot Topics and Popular Papers in Evolutionary Psychology : Analyses of Title Words and Citation Counts in Evolution and Human Behavior , 1979 – 2008. Evolutionary Psychology, 7(3), 348-362. Retrieved from http://www.epjournal.net/filestore/ep07348362.pdf
the Undergraduate Science Librarian 
What a great round-up of research, thanks for writing this!
Bravo Bonnie. I’ll cite you as much as I can! 😉
Very interesting synopsis. I’d never put much thought into the topic although much of what you report makes sense. I was taught to be concise with titles and include key terms to inform potential readers of the content. From now on, I will put much more thought into this… afterall – our scientific endeavors are only valuable if they are well reported and scrutinized by our peers.