Archive

Author Archive

Books I read in 2011

Stereotypically, librarians have a lot of time to read at work.  In reality, any reading we do (even for professional purposes) is typically done in our “free” time.  With the birth of my second daughter this year, that free time was almost non existent, and as a result, I read considerably fewer books this year.  You can view the complete list at Worldcat.org to find each book in a library near you.

Some great non fiction:

Some fiction I really liked:

Some stuff to fill in the time at 3am when you are awake holding a baby and need something to read on your iPhone that you don’t need to think too much about:

And then there is this:

See also: Books I read in 2010 and Books I read in 2009

Categories: personal

Some fun for the holidays

December 23, 2011 1 comment
National Union Catalog Christmas Tree

National Union Catalog Christmas Tree. Who said print was dead? Image courtesy of Flickr user shawncalhoun

For your Christmas entertainment:

Happy Holidays!

Categories: humor

Why scholars cite the things they cite – the real reasons

December 21, 2011 2 comments

In my previous post, I took a look at some of the scholarship about why certain articles are cited more than others.

I feel bad, because by focusing on all of the little things that correlate with citation rate, I didn’t talk about the substantive aspects of how a citation is used.

Sometimes a citation is used to simply say "I agree." Other times it may be used to say "You're wrong."

Cue the next article I found by R.B. Williams (2011) about the history and classification of citation systems in the biosciences.

This was an exciting article to read for two reasons.  First, I had been looking for some information about the history of various citation styles for a while.  (It isn’t easy.  Try Google-ing “history of citation styles”).

Second, the article made me aware of the scholarship about how citations are actually used within scientific documents.  I am particularly drawn to the questions posed by Moravcsik and Murugesan back in 1975.

  1. Is the reference conceptual or operational? In other words, is the reference made in connection with a concept or theory that is used in the referring paper, or is it made in connection with a tool or physical technique used in the referring paper? The distinction is not meant to be a value judgment, and is not to be taken as synonymous with judging the importance of the paper referred to.
  2. Is the reference organic or perfunctory? In other words, is the reference truly needed for the understanding of the referring paper (or to the working out of the content of that paper), or is it mainly an acknowledgment that some other work in the same general area has been performed?
  3. Is the reference evolutionary or juxtapositional? In other words, is the referring paper built on the foundations provided by the reference, or is it an alternative to it?
  4. Is the reference confirmative or negational? In other words, is it claimed by the referring paper that the reference is correct, or is its correctness disputed? Incorrectness need not be claimed through an actual demonstration of an error in the paper referred to, but could also be established, for example, through inferior agreement with experimental data.

First, these questions have a real importance when we start thinking about the ways in which citation metrics don’t necessarily get at the importance of scientific work.
And second, I think there is some potential in these ideas to help students when they write term papers and cite their sources.

Traditionally, I teach students that they need to cite their sources in order to acknowledge the scholarly work of others.  I talk about the implications of not citing something (it was your own idea, its common knowledge, its plagiary), but I don’t really go into more detail about why you might cite something.

By breaking down the purpose of a citation explicitly, as these questions do, perhaps we can better prepare students to effectively use the research articles they find in their term papers and projects.

Now, I’m no expert on teaching writing.  But the best term papers do an effective job of integrating the various sources they find into a cohesive narrative.  Perhaps we could be more explicit about how this is done, and perhaps these ideas can help the students envision what their citations and their term paper might look like.  Perhaps.

References

Moravcsik, M. J., & Murugesan, P. (1975). Some Results on the Function and Quality of Citations. Social studies of science, 5(1), 86-92. Sage Publications. Retrieved from http://sss.sagepub.com/content/5/1/86.full.pdf

Williams, R. B. (2011). Citation systems in the biosciences: A history, classification and descriptive terminology. Journal of Documentation, 67(6), 995-1014. doi:10.1108/00220411111183564

Categories: citations

Will this post be cited more often? Non-content factors that influence citation rates.

December 20, 2011 7 comments

For many researchers, the citation is a make-or-break concept.  Most ranking algorithms use citations to determine a journal’s influence or impact.  Publication in “high impact” journals is often the key to tenure and promotion, and the number of times an article has been cited is often widely touted in tenure and promotion packets.

Image courtesy of Flickr user futureatlas.com

With careers, funding and much else riding on citation, it would be useful for scholars and librarians to know why a particular item gets cited.  We’d all like to think that the only reason an article is cited is because it’s content is relevant (and more relevant than other items) to the study at hand.

Unfortunately, there is some evidence to suggest that other, non-content, factors influence the likelihood of an item being cited.

Big caveat: The quality of these studies is highly variable and their results are sometimes contradictory.  Correlation does not equal causation.

Nevertheless, most of the non-content factors influencing citation rate relate to article discoverability.  You can’t be cited if you can’t be read, and you can’t be read if you can’t be found.  How likely is an article to be found in a database?  Was the article discussed in a newspaper or other popular science forum?  Does the title clearly explain what the article is about (and make you want to read more)?  Is the article already connected to a wide circle of readers via multiple authors or large universities?  While there are classic examples of important scientific publications published in obscure journals, those are the exception and not the norm.

So, in no particular order, here are a few things that folks suggest might influence how often your article is cited:

A lot of research has looked into various aspects of article titles on subsequent citations.

  • Type of title – In an interesting study looking at article titles from PLoS journals, Jamali and Nikzad (2011) wondered if the type of article title affected the citation rate of an article.  In general, they found that article titles that asked a question were downloaded more but cited less than descriptive or declarative titles.  Interestingly, Ball (2009) found that the number of such interrogative titles have increased 50% – 200% in the last 40 years.
  • Length of title – Jamali and Nikzad suggest that articles with longer titles are downloaded and cited less, and Moore (2010) in a quick study found no correlation. However, Habibzadeh and Yadollahie (2010) suggested that longer titles are cited more (especially in high impact factor journals) and a positive correlation between article title length and citation rate was found by Jacques and Sabire (2009).
  • Specific terms – Disciplinary abbreviations (very specific keywords) may lead to more citations (Moore, 2010), where as articles with specific country names in the title might be cited less (Jacques and Sabire, 2009).
  • Humorous titles – To my disappointment, a study of articles with amusing titles in prestigious psychology journals by Sagi and Yechiam (2008) found that these articles were less likely to be cited than other articles with unfunny articles.  Since funny titles are often less descriptive of the actual research, these articles could be more difficult to find in databases.

Articles with funny titles, like this one featured on Discover's Discoblog, may not be cited as much as others.

Positive Results – There is strong evidence to suggest that positive results are much more likely to be submitted and published than negative results.  It seems as though positive results are also more likely to be cited.  Banobi et al. (2011) found that rebuttal articles (either technical reports or full length articles) were less likely to be cited than the original articles, i.e. the articles with positive results were more likely to be cited.  This correlates well with the results of Leimy and Koricheva (2005) who found that articles that successfully proved their original hypothesis were more likely to be cited than articles that disproved the original hypothesis

Number of authors – Leimu and Koricheva (2005) found a positive correlation between the number of authors and the number of citations in the ecological literature, while Kulkarni et al. (2007) found that group authorship in medical journals increased citation counts by 11.1.  However, a blog post by Moore (2010) suggested that isn’t wasn’t the number of authors that were important, but their reputation.  A recent study of the chemical literature that was able to account for article quality (as measured by reviewers rating) found a correlation with author reputation but no correlation to the number of authors (Bornmann et al. 2012).

Industry relationship – Studying medial journals, Kulkarni et al. (2007) found that industry funded research that reported results beneficial to the industry (i.e. a medical device that worked or a drug that didn’t show harmful side effects) was more likely to be cited than non-instustry funded, negative research.

Data sharing – Piwowar et al. (2007) found that within a specific scholarly community (cancer microarray clinical trial publications) free availability of research data let to a higher citation rate, independent of journal impact factor.

Open Access – Lots of studies have been done with mixed results.  A slightly higher number of studies seem to suggest that open access leads to higher citations (See the excellent review article by Wagner (2010)).

Popular press coverage – It makes intuitive sense that journal articles spotlighted by the popular press might be cited more, but this is difficult to prove.  Perhaps the press is merely good at identifying those articles that would be highly cited anyway.  Phillips et. al (1991) were able to take advantage of an interesting situation when the New York Times went on strike in 1978 but continued to produce a “paper of record” that was never published.  Phillips et. al. (1991) found that items written about in the “paper of record” but not published were no more likely to be cited than other articles.

Length of your bibliography – A 2009 study by Webster et al. (2009) suggests a correlation between the length of an articles bibliography and the number of times it is later cited.  They suggest a “I’ll cite you since you cited me” mentality, but online commentators suggest that this is merely a specious relationship (See Corbyn, 2010, and comments therein).

 

So, if you want to publish a paper that gets the highest number of citations, what should you do?  Do your study with a large number of prestigious co-authors.  Submit your long article containing positive results and a big bibliography to a open access journal.  Say something nice about a pharmaceutical company.  Share your data and get the New York Times to write about it.

Oh, and it might be useful to have some interesting and solid science in there somewhere.

 

Really Long Bibliography:

Ball, R. (2009). Scholarly communication in transition: The use of question marks in the titles of scientific articles in medicine, life sciences and physics 1966–2005. Scientometrics, 79(3), 667–679.  Retrieved from: http://www.akademiai.com/index/UH466Q5P3722N37L.pdf

Banobi, J. A., Branch, T. A., & Hilborn, R. (2011). Do rebuttals affect future science? Ecosphere, 2(3), art37. doi:10.1890/ES10-00142.1

Bornmann, L., Schier, H., Marx, W., & Daniel, H. D. (2012). What factors determine citation counts of publications in chemistry besides their quality? Journal of Informetrics, 6(1), 11-18. Elsevier Ltd. doi:10.1016/j.joi.2011.08.004

Corbyn, Z. (2010). An easy way to boost a paper’s citations. Nature. Nature Publishing Group. doi:10.1038/news.2010.406

Habibzadeh, F., & Yadollahie, M. (2010). Are Shorter Article Titles More Attractive for Citations? Cross-sectional Study of 22 Scientific Journals. Croatian Medical Journal, 51(2), 165-170. doi:10.3325/cmj.2010.51.165

Jacques, T. S., & Sebire, N. J. (2010). The impact of article titles on citation hits: an analysis of general and specialist medical journals. JRSM short reports, 1(1), 2. doi:10.1258/shorts.2009.100020

Jamali, H. R., & Nikzad, M. (2011). Article title type and its relation with the number of downloads and citations. Scientometrics, (49), 653-661. doi:10.1007/s11192-011-0412-z

Kulkarni, A. V., Busse, J. W., & Shams, I. (2007). Characteristics associated with citation rate of the medical literature. PloS one, 2(5), e403. doi:10.1371/journal.pone.0000403

Leimu, R., & Koricheva, J. (2005). What determines the citation frequency of ecological papers? Trends in ecology & evolution, 20(1), 28-32. doi:10.1016/j.tree.2004.10.010

Moore, A. (2010). Do Article Title Attributes Influence Citations? Wiley-Blackwell Publishing News. Retrieved December 16, 2011, from http://blogs.wiley.com/publishingnews/2010/09/02/do-article-title-attributes-influence-citations/

Phillips, D. P., Kanter, E. J., Bednarczyk, B., & Tastad, P. L. (1991). Importance of the Lay Press in the Transmission of Medical Knowledge to the Scientific Community. The New England Journal of Medicine, 325(16), 1180-1183.  Available via: http://www.ncbi.nlm.nih.gov/pubmed/1891034

Piwowar, H. A., Day, R. S., & Fridsma, D. B. (2007). Sharing detailed research data is associated with increased citation rate. PloS ONE, 2(3), e308. doi:10.1371/journal.pone.0000308

Sagi, I., & Yechiam, E. (2008). Amusing titles in scientific journals and article citation. Journal of Information Science, 34(5), 680-687. doi:10.1177/0165551507086261

Wagner, A. B. (2010). Open access citation advantage: an annotated bibliography. Issues in Science and Technology Librarianship, (60). Retrieved from http://www.istl.org/10-winter/article2.html

Webster, G. D., Jonason, P. K., & Schember, T. O. (2009). Hot Topics and Popular Papers in Evolutionary Psychology : Analyses of Title Words and Citation Counts in Evolution and Human Behavior , 1979 – 2008. Evolutionary Psychology, 7(3), 348-362. Retrieved from http://www.epjournal.net/filestore/ep07348362.pdf

Categories: citations

What determines the value of academic publications?

And can librarians, scholars and publishers agree about it?

Image courtesy of flickr user 401K, 401kcalculator.org

By value, I don’t just mean the impact factor and other metrics, or even the general prestige of a journal as measured by gut feeling. I mean the value of a publication (a single article, an entire journal, every journal a publisher publishes) in fiscal terms. Dollars and cents. Moolah. Benjamins.

Perhaps we can say that a more highly ranked journal (impact factor, eigenfactor, etc.) might be worth more in dollars. Some publishers would certainly like this to be true. Higher quality equals higher cost. After all, you want the $100 bottle of wine to taste better than the $5 bottle of wine. But this doesn’t seem to be how it works in reality, at least in some disciplines. Bergstrom and Bergstrom (2006) did a study of ecology journals, and examined cost versus impact factor.  In general, they found that there was no correlation between the two.  [Yes, yes, I know. Impact factor is just one poor way of measuring value.]

Perhaps value has something to do with reliability? One way to look at reliability is to examine retractions. Fang and Casadevall (2011) have shown that typically, high impact journals have more per-article retractions because they are at the cutting edge of research (also see this Retraction Watch post). Because they want to get things out fast, mistakes are sometimes made. Nature, Cell and Science all have a relatively high Retraction Index (see Fang and Casadevall, 2011). But does this make these journals any less valuable overall?

With the number of open access options increases, publishers (especially for profit publishers and academic societies that act like for profit publishers) make the argument that their editing, copy editing and page preparation services add significant value to their publications.  How much should should these services add to the total value of the publication?  [Hint: not as much as the publisher would like.]

Then again, perhaps the value of a publication has less to do with the content and more to do with the audience. For example, the New England Journal of Medicine is probably more useful and valuable to a medical school than it is to my small liberal arts college, and more valuable to me than to a similarly sized school without a biology program.

When publishers assign a value to their publications, they typically take into account the size of the school or the types of degrees they award. Larger schools often pay more money for access to the same resources. Unfortunately, this doesn’t always take into account the details of who those folks are. Two schools with 5000 students will pay the same amount for a resource in chemistry, say, even though one school has 100 chemistry majors each year and the other has 10.

As journal costs keep rising, institutions must continuously evaluate value – does this journal provide enough value to my institution to justify the costs?

No matter how good the $100 bottle of wine is, I’ll need to keep drinking the $5 stuff. Or maybe the $10 stuff for Christmas.

References

Bergstrom, C. T., & Bergstrom, T. (2006). The economics of ecology journals. Frontiers in Ecology and the Environment, 4(9), 488–495. Retrieved from http://www.esajournals.org/doi/abs/10.1890/1540-9295(2006)4[488:TEOEJ]2.0.CO;2

Fang, F. C., & Casadevall, A. (2011). Retracted science and the retraction index. Infection and immunity, 79(10), 3855-9. doi:10.1128/IAI.05661-11

Categories: publication

Trust but verify the results of automatic citation creation tools

November 30, 2011 1 comment

Almost all academic databases these days will allow you to export a properly formatted citation (APA, MLA, etc.) for a book or journal article within that database. This is a wonderful feature for undergraduates that saves a lot of really annoying formatting. It is especially helpful for eliminating the annoyance of re-arranging author first names and last names and putting in appropriate punctuation.

Unfortunately, it doesn’t always come out perfectly.

For example, the citation database Scopus regularly produces a citation indicating that an article is “Available from www.scopus.com” which is completely incorrect.  Just the citation is available from Scopus, the full text of the item is found elsewhere.

So in my library instruction sessions I regularly encourage students to double check the results of these citation generators (in databases, in web services like EasyBib and in programs like Mendeley and EndNote).

Because this is what happens when you don’t look things over:

Bad citation from the tumblr blog "Shit my Students Write"

So take a minute or two to look over your bibliography – you don’t want to look silly.

Categories: citations

Digital Object Identifiers for 17th century publications

The Royal Society of London recently announced that the historic archive of the Philosophical Transactions of the Royal Society will now be openly available (Happy Open Access Week!) 

Philosphical Transactions is typically regarded as the first scientific journal and has been in continuous publication since it started in 1665. (A french journal, the Journal des sçavans started publication three months prior to the Philosophical Transactions, but since it appealed to a wider audience and included a larger percentage of book reviews, many do not consider it the first real scientific journal).

We’ve had access to this archive for a while now via JSTOR, and I love having the ability to see the very beginnings of the scientific journal article.

What intrigued me when I started digging into their now-open archive was the delightful juxtaposition of the 1665 publication date and the modern DOI.

Since these historical documents are available online, they are digital objects, and assigning DOIs makes a lot of sense.  It also makes each individual article much easier to find.

So, hurrah for Open Access, hurrah for easy access (not necessarily the same thing), and hurrah for An Account of a Very Odd Monstrous Calf:

Boyle, Robert. 1665.  An Account of a Very Odd Monstrous Calf.  Philosophical Transactions. 1: 10. doi: 10.1098/rstl.1665.0007

Boyle, Robert. 1665. An Account of a Very Odd Monstrous Calf. Philosophical Transactions. 1: 10. doi: 10.1098/rstl.1665.0007

Categories: citations

Tracking down a citation shouldn’t be this hard

October 20, 2011 5 comments

Tracking down a citation you already have should be a relatively simple task.

A colleague of mine asked for help the other day tracking down a citation.  A variety of circumstances made it anything but straightforward and served to remind me about some of the confusing parts of the scholarly communication system (and that I really love my job).

A student had approached the reference desk looking for a citation to this article:

Tan, D. X.; Chen, L. D.; Poeggeler, B.; Manchester, L. C.; Reiter, R. J. (1993). “Melatonin: a potent, endogenous hydroxyl radical scavenger”. Endocrine J 1: 57–60.

The student had found the citation via the Wikipedia entry for Melatonin.  My colleague started out with the usual process – look up the journal, find the right volume and go from there.  Except when you look up Endocrine Journal, you find that the volume number doesn’t match the year, nor are there any articles with a similar title in the publication.  Author searches in the same journal also yield nothing.

Since the citation came from Wikipedia, it’s seemed probable that there was an error.  So she did a search on Google and Google Scholar to try to find a correct citation.  Neither search turns up the article, but Google Scholar indicates that the article has been cited over 1000 times!  The student found another article by some of the same authors on the topic and was content, but my colleague still wanted the answer.  With other students waiting for reference help, she sent the question along to me.

I was checking my email after my kids went to bed and thought I’d poke around a little to see what I can find.  I re-did the searches my colleague did so that I understand the problem. Theoretically, the article has to exist, since it has been cited so many times.  So why couldn’t we find it?  I tried Google Scholar, PubMed, Scopus and found nothing (we don’t have Web of Science here).  I searched for additional publications by the same authors but I still didn’t find anything close to this one.

So I started looking for similarly name publications.  The journal Endocrine Journal is published by the Japan Endocrine Society and the years don’t match up, so perhaps the abbreviation refers to something different?  I located a journal called simply Endocrine (try finding that one in a Google search!) published by Springer.  This started to look promising because the first volume of of Endocrine was published in 1993, just want we want.  But this volume isn’t available on the publisher’s website, so I couldn’t confirm my suspicions.

If Endocrine is the journal we want, why can’t I find it indexed in a database?  I checked indexing information.  PubMed only started indexing it in 1997.  Scopus started indexing it in 1993, but only with the fifth issue, and we need issue 1.  And Google Scholar won’t have it (other than the citation) because it isn’t on the Springer website or in PubMed.

I start to think that the citation really refers to an article in Endocrine, not Endocrine Journal.  But Scopus has over 1000 folks citing Endocrine Journal.  It seems unlikely that so many people would make the same error.

I stayed up past my bedtime having fun tracking this down.  I emailed my thoughts to my colleague and I wondered if perhaps Web of Science indexed this item from issue 1.

The next day, we asked a colleague at another institution to do a quick search for us in Web of Science.  No hits on the article title.  Perhaps Web of Science didn’t index it from issue 1 either, or perhaps I’m just wrong (it’s been known to happen).

Then I checked Ulrich’s guide to periodicals.  We have it in print here, and the brief entry illustrates the missing piece of our puzzle.

The entry in Ulrich

The entry in Ulrich's clearly indicated this journals former title, a fact that is missing from the publisher's website.

From 1993 to 1994 there were two Endocrine Journals!

For a brief period of time (<2 years), Endocrine called itself Endocrine Journal.  Perhaps they discovered the Japan Endocrine Society’s Endocrine Journal as the internet was making international collaboration easier.

Since I found the original ISSN (0969-711X), I submitted an ILL request to confirm my thoughts.  Sure enough, here’s the article masthead, but with Macmillan Press as the publisher, not Springer.  The early issues available on the Springer website have Stockton Press as the publisher in 1995.  It seems to have changed publisher several times.

Article screenshot

Screen shot of the PDF file I received via ILL. Note the publisher at the top.

What’s the moral of this story?

  1. Journals really need to select unique names.  (Do new journals think about Google-ability of their names?)
  2. I picked the right profession because I had fun chasing this down.
  3. Given my difficulty tracking this down, I have to ask: How many of the 1000 folks that cited this article actually tracked it down?  I bet there are some who never laid eyes on it.

More importantly, it can be very easy for valuable information to disappear entirely.  We live in an era of information overload.  Yes, people have been saying the same thing since the invention of the printing press, but these days it isn’t a matter of finding any information, it is a matter of sorting to find the right information.  And even today, an item published just before the explosion of online scholarly information could almost disappear.  Although it may seem like it, not everything is available in Google.

Categories: citations

iPhone Apps for Research and Collaboration

October 19, 2011 2 comments

I recently posted an entry on my library’s blog about some iPhone apps that might be useful to undergraduates for research and collaboration.  I thought some readers here might find the same information useful.

You may also be interested in a list of more science-literature-related iPhone apps that I published last year.

Check out the following free apps to help you search the literature, cite your sources, and organize your work.

iPhone Apps
iPhone apps for research and collaboration

Ebsco Databases – Ebsco provides access to a large number of databases via one app (ERIC, Georef, American History and Life, MLA International Bibliography, Business Source Complete, Academic Search Complete and lots of others).  Because access to these databases is paid for by the library (with your tuition dollars), you need to log in to Academic Search Complete via your library’s website first.  At the bottom of the screen you’ll click on a link that will send an email with an activation code.  After downloading the app, open your email on your phone and click on the link.  You will then have 9 months of access.  I’ve found this process to be pretty simple and easy – no need to log in every time.  The app will connect you to full text articles within the Ebsco databases, and even Geneseo’s “Get it” service (our version of OpenURL) for articles found elsewhere.

SciVerse Scopus Alerts – A search app for the interdisciplinary database Scopus.  I’ve reviewed this app before, and there hasn’t been a major update since then.  This app can do keyword searching, citation tracking, and alerts for the science and social science literature.  Scopus is an outstanding database, but the app has some issues.  The biggest problem is getting it to work.  You need to remember your Scopus username and password (not your Geneseo username), and even then there can be trouble.  While the tech support is responsive, it just isn’t as easy to get started as the Ebsco app above.

Evernote – I recently started using this piece of software on my computer for note taking during meetings and lectures, lesson planning and writing.  I am in love with its simplicity and universal usefulness.  Take class notes on your computer, then download the iPhone app to access them anywhere.  Record voice notes on your phone and automatically sync them to your laptop.  Take pictures with your phone and insert them into the notes you’ve already started, or start a new note.  The iPhone app syncs with the desktop application so that you never have to guess where a certain piece of information is.  Share notes with others via shared notebooks or simple weblinks.  I love this app.

Dropbox – Along with the Dropbox website, this tool allows you to easily share files among friends (with shared folders), or between your computer and phone.

EasyBib – An app from the popular website.  This app allows you to scan the barcode of a book and create a formatted citation (which you will, of course, check against the style manuals for accuracy).

Merriam Webster Dictionary – There are lots of dictionary apps out there.  This one is free, and has a nifty voice search function.

Mendeley – This app works with Mendeley Desktop and the Mendeley website.  It allows you to store and organize your PDF journal articles and book chapters.  It’s like iTunes for journal articles: Mendeley will organize your folders for you and you can create folders (playlists) of articles.  You can share those folders with others to help you collaborate on group projects.  The desktop version integrates with Microsoft word to help you cite your sources.  This mobile app allows you to access the journal PDFs you have synced to the web, as well as the ability to search your personal library.  This is one of a few applications that is always open on my laptop, and I love the ability to quickly look things up on my phone when I am away from my computer.

Since I don’t have an Android phone, I can’t comment on the availability or usability of these apps on that platform.  Perhaps in another post.

What apps do you use to get your work done?

Categories: Uncategorized

Keeping track of it all

Along with every other department on campus, libraries are under increasing pressure to evaluate their services – everything from student learning outcomes to expenditures.

Tally marks

Are your reference statistics a comedy or a tragedy? Image courtesy of Flickr user aepoc

Assessing the value of libraries in these areas requires the collection of lots of information.  Data of all kinds needs to be collected, analyzed and shared.  So what data do we collect, and where do we store it?

We have lots of silos for relevant information here in my library, and none of us are convinced that we are doing things in the best way possible.  Our collection of statistics related to reference and research help services provides one example.

The most obvious place where this happens is the reference desk.  To keep statistics about what happens here we use LibStats to record:

  • the question itself,
  • the format (phone, walk-up, IM)
  • the patron (student, community member, faculty member)
  • how long it took to answer

But our research help doesn’t end at the reference desk.  One of the big services we provide is research consultations by appointment.  Students (and faculty or community members) can request an appointment and their request will be routed to the most appropriate librarian.  (No one wants me answering in-depth research questions about primary sources in 17th century European history, for example.)

These requests come via an online form that dumps information into a home-grown MS Access database.  For this kind of appointment-based research help we collect the same information recorded for reference desk questions, but also information about the student and the course the project is related to.

But our research help comes in other forms, too.  We have an email-based ask a librarian service, and we all get email questions directly from students and faculty.  At this point we aren’t very good at recording this type of information.  What system should we use?

We also aren’t very good at recording questions that come directly to us from faculty, either via email, phone or in person.

And I haven’t even started to discuss the challenges of assessing the student learning outcomes associated with research and reference help services.

As a result of all this, it is difficult to get one complete picture of our involvement in research across campus.  It’s something we are currently working to resolve.

And the biggest question that will influence how we do this is

“What do we want to do with this information?”

Change our services?  Change our staffing levels? Merely collecting the data won’t be of use to anyone.  The answer to these questions will influence the type of data that we  collect and the tools we use to collect it.

And once we figure out all that, then we just need to remember to record everything.

Categories: assessment, librarianship
Follow

Get every new post delivered to your Inbox.

Join 43 other followers