Skip to content

Assessing Student Learning During Reference Transactions

Last week I had the pleasure of attending and presenting at the annual conference for SUNY librarians at the Fashion Institute of Technology in New York City (yes, FIT is a SUNY school!).

With my exceptional colleague Kim Hoffman, we discussed a small project we did in the Fall 2011 semester to try to assess what students learned at the reference desk.

Our abstract for the presentation and our actual slides are below.  You can also view the slides on Slideshare.net to see the speakers notes.

Going Beyond Anecdotes: Assessing Student Learning During Reference Transactions

Reference services comprise one of the most important teaching opportunities within academic libraries.  While we typically assume that students learn from these interactions, we rarely have evidence to demonstrate what students actually learn.  Librarians at many institutions track the skills taught via reference statistics gathering programs, but we rarely ask students what they find most meaningful.

At SUNY Geneseo, we wanted to know what students were learning via reference transactions, beyond typical counts of reference questions or user satisfaction surveys.   These reference transactions occur in several settings, including at the reference desk, during scheduled reference consultations, and through impromptu questions at various locations.  Building on assessment techniques such as the One Minute Paper traditionally used in library instruction settings, students were given a survey after each reference transaction that simply asked “What did you learn today from your meeting with the librarian?”

In order to categorize responses, librarians developed a list of commonly taught concepts, skills, and tasks seen via reference services and library instruction.  Student responses were assigned one or more items from this list of concepts, allowing us to easily evaluate which skills were most frequently reported.

While this survey explores which concepts students report learning, it does not measure their actual mastery of the skills reported and is therefore an incomplete examination of student learning at the reference desk. Despite these limitations, this study offers a useful improvement to standard reference assessment efforts, typically based on assumptions and anecdotes.

A new venture: the Scientific American Blog Information Culture

Some readers of this blog may be interested in  a new blog I will be co-authoring on the Scientific American Blog Network called Information Culture.

I will be co-blogging with Hadas Shema, an information science PhD student who has previously blogged on some fascinating stuff on her blog, Science Blogging in Theory and in Practice.

As a result, my posting frequency here will probably drop, but hopefully won’t disappear altogether.  There are some things I want to talk about that just might not be of interest to the Scientific American crowd. But we’ll see.

We will be talking about many of the same things that I talk about here: open access, scholarly publishing, sources of frustration in finding information, etc.

Please have a look!

Take the money and run

With library funds decreasing, for-profit publishers want your science research money.

To those in the know, this isn’t big news. Publishers have been hinting about it for some time.

Keys and money

For-profit (and some non-profit) publishers want more money, while keeping their content locked down and controlled. CC image courtesy of Flickr user Images_of_Money.

But a recent analysis report (PDF) has been making the rounds of the twitter-verse today, making a few things more explicit. Reed Elsevier: Is Elsevier Heading for a Political Train-Wreck? is an investment analysis from Claudio Aspesi and others at Bernstein Research:

Most important, at a time when budgets of academic libraries look likely to be constrained for years to come in many countries, Elsevier’s growth will increasingly depend on its ability to secure funding earmarked for general science research, instead of library funding.

This fascinating report, lays out the challenges facing Elsevier in an age of boycotts, open access, and increasing researcher awareness of the costs of scholarly publishing.

The report also focuses on Elsevier’s culture of control of their content. From limits on open access, licensing restrictions and text mining restrictions Elsevier wants to control who does what with their content. They want to take an ever increasing share of library budgets, then the research grant money, and they want to control what you do with the information you license from them. The investment analysts state:

We continue to be baffled by Elsevier’s perception that controlling everything (for example by severely restricting text- and data mining applications) is essential to protect its economics.

Recently, it seems like every year I talk to faculty in my departments and say “We have to make some tough choices due to flat budgets and increasing journal costs. What would you like to cancel?” Every time librarians do this, faculty become a bit more aware of the economics of scholarly publishing.

Elsevier isn’t the only publisher facing these challenges, and they might not even be the baddest apple in the group, but they are very big (my library pays more for their content than anyone else’s), and their actions are drawing the attention of the scholarly community.

It will be interesting to see how this develops. Stay tuned!

Publishers, Hyperbole, and the “Don’t subscribe” pricing model

Commercial publishing is no stranger to hyperbole. “Essential research for your institution.” “Best information resource available.” “Exclusive time-limited offer.”

But I recently came across an interesting case of publisher hype. Multi Science Publishing, publisher of many mid-range scientific journals, recently sent an email to an email discussion group, touting its new pricing model, “Pay only for Usage.” Their tag line is “Don’t subscribe.”

The email claimed that announcements of the new model had caused “quite a stir” and the author of the email, a W Hughs, the “director” of Multi-Science Publishing, suggested that he hadn’t seen “anything like” the scale of libraries response to the new plan.

That stuck me as quite interesting, because a search for information about this new plan yields little information about the plan.  The most prominent resources are a link to the email discussion group archives and a single blog post, both dated March 27.  I can’t seem to fins information about the package on the publishers website.  The only stir on twitter I can find are tweets from @billhughes6 directed towards libraries:

In short, I’m not convinced that this announcement has caused a stir at all.

And so you may ask, is the new model deserving of the hype?

The pricing model they’ve declared revolutionary is simple: libraries sign up for access and pay $5.00 for each article their users download. For folks in the Sciences, the $5.00 per article price point is less many publishers charge, and it’s even less than the interlibrary loan fees that libraries might have to pay.

It isn’t a bad deal. The big part seems to be the unmediated bit – users get direct access to the article, without having to request that the library buy it for them (although some new methods of making these requests have reduced the time needed to get the article to you, you still have to ask.)

So, how does this compare to other journal packages out there? That’s a difficult question to ask, because libraries don’t pay per article. In same cases, we can easily pay $1, $10 or $40 an article, depending on the journal, publisher and package.

Libraries may also be reluctant to sign up for a plan that will make it difficult for them to budget their expenses.

I like to see some experimentation in journal pricing models, because the status quo isn’t really helping anyone (except company shareholders).

I’d be less cynical about the plan if the publisher had simply promoted the plan, without informing us of how much “stir” and “excitement” it had already caused – earn the buzz for your new service, don’t just say it exists.

Open access challenges for small scholarly societies

I am very much in favor of open access.  I believe it is the natural extension of the scientific enterprise.  Scientists no longer record their results in code, or disseminate them via cryptic anagrams.  Instead, the work of scientists is shared with others so that they may, in turn, make new discoveries.

Creative Commons image courtesy of PLoS, available via Wikimedia Commons

Yes, this is idealistic, but I’m okay with that.

As a result, I have no hesitations in pushing the big for-profit publishers towards greater rights for authors and more open access options, and I applaud the effort behind the Cost of Knowledge  boycott of Elsevier.  And just being a not-for-profit scholarly society will not make me sympathetic to high subscription costs, aggressive price increases and restrictive copyright practices (like the American Chemical Society).

But for smaller scholarly societies, I can see how the open access movement has caused a lot of soul searching and a wide variety of opinions and options.

For small scholarly societies, subscriptions to their scholarly publications can make up a large portion of their operating budgets.  Moving to an open access model may mean the loss of some of this revenue, and society members may question whether an author-pays model of open access publication will be able to offset the cost of publication.  On the other hand, many societies may see a greater fulfillment of their mission by expanding open access options.

Happily, I am seeing more small scholarly societies embrace various aspects of “openness” in their publications.  The Ecological Society of America demonstrates some interesting examples of branching out and offering more open access options.

First, they recently started a new open access journal, Ecosphere.  The new journal conforms to what we tend to expect from “gold” open access publishers: online only, author fees for accepted manuscripts, and authors retain copyright of their articles.  The recent ESA annual report  suggests that they have been pleasantly surprised but the success of the new journal.

Second, although ESA requires transfer of copyright to the society for their other publications, they do grant the right to post a copy of the article on the authors personal homepage or institutions website.  This is “green” open access, an option that more researchers need to take advantage of.

Finally, their journal Ecology provides and interesting example of a hybrid publication.  Ecology is available via subscription and publishes a wide variety of article formats, including brief reports that are “expected to disclose new and exciting work in a concise format.”  Several reports are published in each issue, and all are open access.  As a result, a certain portion of each issue is freely available.

As scholarly societies and other publishing entities come to terms with new expectations for scholarly publishing, I expect that more societies will experiment with a variety of open access options.  A three year old report from SAGE suggests that this will happen. I’m looking forward to seeing what folks come up with.

Tracking down articles from science news stories

Readers of this blog may be interested in a piece I wrote for the Scientific American Guest Blog, How to: Track down journal articles cited in news stories (when they don’t link directly).

Many blog posts will link directly to a version of the original article, but many news sources often have a policy of not linking to the original source. Even when a blog links directly to the original article, you may not be able to read the it without paying. But there are steps you can take to find the original article, and to find a version of it you can read.

Read more.

In which container is the journal article I need?

The other day, I got an email from a faculty member.  A scholarly society he is a member of just announced that their journals would now be available in JSTOR.  He went straight to JSTOR to look them up, only to see that he didn’t have access.  He promptly sent me an email saying, essentially, “What’s up with this?  Shouldn’t we have access?”  (Although his actual email was more eloquent).

Containers

In which container is the journal article I need? CC image courtesy of flickr user s_volenszki

In fact, we don’t have access, and it would cost us an additional $1000 to have access to those journals via JSTOR.

For non-librarian types (students, faculty, everyone else), there isn’t always a clear understanding of how they have access to information.

In the case of JSTOR above, most folks don’t understand the difference between the platform and the content (and quite frankly, they don’t really need to).  In this case, JSTOR is simply a platform for delivering journal articles.  You have to buy the content, and that tends to come in specific chunks.  In my library, we subscribe to several of the packages that JSTOR offers, and we have current access to some of the journals that are available via that platform.

But just because we have access to some content on JSTOR doesn’t mean we have access to everything.  The same can be said of other platforms like ScienceDirect from Elsevier, or Project Muse (for any humanities folks out there).

In much the same way, it can be difficult for folks to understand that libraries don’t always have access to journal articles direct from the publisher’s website.  We have access to a lot of journals via third party aggregators, like Proquest or the Ebsco packages.

For example, a student or researcher wants an article from the current issue of the Journal of Parasitology, and goes directly to the journal homepage.  When they get there, they encounter a paywall, asking for $20 for access to that article.  The student or researcher might think that they either have to fork over the money or move on to a different article.

While the student searches for a new research topic, a PDF of this exact article is sitting in our “MEDLINE with Full Text” database ready for them to download.  We’ve already paid for the content, just not through the journal website. Our current access to this journal is via a different platform.

In library instruction sessions we try to teach students to go through the library homepage to check on journal access, but it isn’t always the most intuitive thing to do.  And some students can go their entire undergraduate careers without seeing a librarian in their classes.  We also teach them to use special links we put into databases that will guide them through the library system (we call ours “Get It,” the generic term is OpenURL), but the databases don’t always help us out here.  Some databases provide direct-to-publisher links which, as we’ve seen, don’t always lead to the content.

Is this confusing?  Yes.  Could it be simpler?  Yes, but it would require a complete rethinking of the whole scholarly communication system.  Open access, anyone?

Follow

Get every new post delivered to your Inbox.

Join 78 other followers