Skip to content

The results are nice, but what I really need to know is HOW you did it

For the past few days I’ve been at the 2014 Library Assessment Conference in Seattle, WA. It has been a great conference and I’ve got tons to think about and lots to do when I get back to work.

As I listen to all of the presentations one of the biggest things that strikes me is that I wish folks would spend a bit more time on HOW they did the assessment.  I like hearing about results and changes that were made as a result, but what I really need now are methods.

It is common sense to take a method and try it. If it fails, admit it frankly and try another. But above all, try something. - Franklin D. Roosevelt

And so I began compiling a list of the methods that librarians use when we assess our services (including student learning). There are lots of variations of each method, but I’m trying to think of the fundamental methods that assessment folks should learn about:

Data collection:

  • Surveys and tests
    • How to write good questions? When is a survey appropriate (or inappropriate)? What can we learn (and what can’t we learn) from surveys?
  • Focus groups
    • How can we recruit participants? How to structure the conversation? How to use the results?
  • Observations
    • What to look for? How to record your observations?
  • Structured interviews
    • How to write good interview questions? What are good interview techniques?
  • Automatic capture (ILS, ILL, Counter, etc.)
    • What is your library currently collecting? How is it accessed? What are folks currently doing with it?
  • Event capture (not sure what to call this, but I’m thinking of reference stats)
    • What are you currently capturing? How are you using that data?
  • Collecting authentic work (student papers, faculty publications)
    • What to collect? How to encourage faculty and student participation?

Data analysis:

  • Statistical analysis of existing data (ILS, Counter, Ref stats, etc.)
    • I think this is the area that scares librarians the most
    • What types of statistical analysis do we need to know about? How can this help us?
  • Data visualization
    • When is this most appropriate? What kinds of visualizations are the most helpful? What tools should we use?
  • Rubrics
    • How can we develop a good rubric? When should we use such a time intensive method?
  • Content analysis
    • How do we develop coding categories? What software do we use? How to interpret and share the results?
  • Citation analysis
    • What can this tell us? Do we look at student papers or faculty publications? What metrics might be the most helpful?

Obviously, each of these categories contains many variations on the theme, and expecting one person (or all library staff) to know about all of these methods is unrealistic. But we should probably be aware of what methods exist, so that when we need one, we can get some help in applying it.

What other methodologies are libraries using?

John Oliver as an Instruction Librarian

I’ve been a big John Oliver fan from his time at the Daily Show and his weekly podcast with Andy Zaltzman, The Bugle, but I never knew just how effective he could be as an instruction librarian. Just watch this video below as he carefully examines the emerging media form of native advertisements and the problems associated with this un-holy alliance between the business and editorial sides of media organizations.

Update: Just yesterday, Advertising Age published an article about the New York Times shrinking the labeling of it’s Native Advertisements. It is difficult enough for people to recognize the difference real ads and news stories, this makes things much harder.

http://www.youtube.com/watch?v=E_F5GxCwizc

Librarians need bigger egos

Obviously, not all of them.  Some of us have big enough egos and need to tone things down a notch.  I’m not talking about the big egos and so-called “rock star librarians,” but the egos of regular working librarians.

In a 2008 article in Library Journal, Casey and Stephens argue that egos are bad for libraries:

The ego, we concluded, can be a very damaging thing. Inflated. Overbearing. Egos create rules for rules’ sake. Egos complicate procedures and keep good people down. Egos squash good ideas and can take the best of an organization and turn it on itself.

But they really refer to over-inflated egos.  I argue that a healthy, reasonable ego is a good thing. For all of us. Perhaps this is semantics: since the word ego has some pretty negative connotations, maybe I really mean to suggest that librarians need more professional self-confidence or self-esteem.

Because librarians are smart. Damned smart. They are talented, knowledgable, hardworking and willing to go out their way to help others out. If you want to find something out or get something done you should definitely ask a librarian.

But I’ve seen colleagues acquiesce without any discussion to poorly thought out faculty demands regarding library instruction. I’ve seen librarians sit quietly through meetings with bosses or administrators and then provide intelligent, thoughtful criticism after the meeting when the boss isn’t listening.  I’ve heard colleagues at conferences complain about faculty not including them in learning management systems and I find out that they never asked.

What contributes to this quietness, this passivity, this inability to assert ourselves even in the areas of our expertise?

Is it gender? Over 80% of librarians are female, and workplace gender dynamics might come into play.  I’m certainly no expert on this topic, but books like Nice Girls Never Get the Corner Office and Lean In seem to suggest that women need to be more assertive at work and stop confusing “being nice” with asking questions and stating opinions. NPR has an interesting new series called the Changing Lives of Women. As a part of that series, they have created a tumblr project called She Works: Notes to Self encouraging women to share their slogans, affirmations and advice. Many of submitted slogans encourage women to speak up, “Sit at the table and speak up,” and “Don’t be shy. Promote your accomplishments.”  But there are also of slogans encouraging women to be nice or be quiet, “Smile on the outside, tell them off on the inside” or “Work hard and be nice to people,” advice that I’d bet wouldn’t be posted a similar site geared to men.

Sit at the table and speak up

Is it education?  Although librarians often have faculty status, we most often do not have PhDs like most of the rest of the faculty. I routinely call professors by their first name since we are colleagues and that’s what colleagues do these days.  But other librarians routinely call professors “Professor Smith” even when the professor uses the librarian’s first name.  Are librarians intimidated by the title or the degree? Are some folks less likely to state opposing opinions or ask challenging questions?

Is it the library’s place within the institution?  Although we are often faculty, we are different than classroom faculty. No matter how robust our library instruction programs, we sit outside of the classroom and teacher model that serves as the core of most high education institutions. And in a digital world, some faculty start to question the ongoing relevance of the brick-and-mortar library.  Are we stymied by our kind-of-outsider status?

I don’t know what the answer is.  But I’ve met and spoken with lots of librarians, and I know what they are capable of.  They are amazing, articulate professionals with a deep understanding of how folks search for information and the knowledge of what kinds of information is out there. We know about scholarly publishing, instructional design, data resources, pedagogy and a gazillion other things.

Let’s dust off those egos. Let’s make sure other folks know our strengths. Let’s stand up for our accomplishments.

Getting your friends and colleagues to share what they know

One of the things I love about working in an academic library is the steady opportunity to learn about new things.  I learn things when I help students, work with faculty and talk with my colleagues. Over the last couple of years I have worked to organize an informal series of workshops to help librarians and faculty share the things they know with each other.

It started a couple of summers ago, when the newly formed Instructional Design team at my library organized a series of technology workshops.  We each took turns sharing new websites, apps and other tech tools that we liked and used in our work.  I thought this was great, and I loved hearing about all the things my colleagues knew about.

Classroom chairs

CC-BY image courtesy of Flickr user James Sarmiento

Last summer, I wondered if they were going to do the same thing.  At the same time, the Scholarly Communication team in my library was hoping to do some workshops for library staff about things like open access and the Elsevier boycott.

At this point I took over getting things organized, and twisted the arms of my colleagues to put together workshops about things they were knowledgable about.

It was a rather selfish move on my part – I wanted to learn about the things my colleagues knew.

With help from colleagues, we brainstormed things that we wanted to learn about and recruited folks to present on those topics. I wanted the workshops to have an informal feel: I sought out hands-on workshops and discussions more than formal presentations.  I also asked the simple question: why don’t we invite folks across campus to these workshops?  There was no good reason not to, so we sent campus wide emails advertising the workshops that would be over interest to folks beyond the library.

The first summer I organized the workshops, all of the speakers were library staff members.  This year, I asked for workshop topics from our CIT office on campus and a couple of faculty who are doing some interesting things. I’m excited about the workshops they will be presenting.

Now, we are a small institution with a small number of faculty.  I needed to be realistic in terms of my expectations of attendance: we weren’t going to be filling lecture halls.  Attendance at the 2012 workshops varied widely, from a low of 4 to a high of 16 folks from the library and across campus. For our small campus, I was quite happy with these numbers.

At the end of the summer, I sent an evaluation survey to campus faculty and staff and got some great feedback regarding workshops to hold again, ways to improve communication about the workshops and suggestions for future workshops.  One of the less tangible benefits of the summer workshops was the way in which the existence of the workshops (and the emails announcing them) added to the library’s reputation as a group of folks to talk to about scholarly communication issues or some instructional technology issues.

Here are some of the workshops we have lined up for this summer, relevant to staff across campus:

  • What’s In a Name? The Many Facets of the Word ‘Editor’
  • Mendeley
  • Zotero
  • Time Management for Busy Geeks
  • Gmail community roundtable: labels, searches, filters, labs and more
  • Copyright and Creative Commons
  • Trends in peer review: third party peer review services
  • In praise of paper: an open discussion about our favorite paper based tools
  • Introduction to R: Free and open source program for statistics and data analysis
  • Reading your Copyright Transfer Agreement
  • Video hosting and sharing with Ensemble
  • Managing your online professional identity
  • Open Educational Resources
  • Instant Response System

I’ll be leading a couple (Mendeley, Trends in Peer Review) and attending almost all of them.  I’m excited to learn about some interesting things from my smart and talented colleagues.

Does your library have a professional development program?  How do you facilitate the exchange of knowledge between library staff?

Setting up research consultation appointments using Doodle’s MeetMe page

This afternoon I saw positive results of a little experiment I set up using the find-a-good-meeting-time site Doodle and its MeetMe feature.

You see, I don’t have office hours. Sure, there are lots of hours when I am in my office, but unlike teaching faculty, they are never the same from week to week.  This week I am available on Friday at 2pm, next week I’m not. This can make it complicated to set up meetings.  I can rely on viewing my colleagues calendars in Google Calendar, but making appointments with students can be more complicated.

Most of them don’t use Google Calendar, and it can take a lot of emailing back and forth to find a mutually agreeable meeting time.  I make my schedule available online, but most of them don’t think to look it up.

My profile from our Subject Guides pages

My profile from our Subject Guides pages

A couple of months ago, I put a link to the MeetMe service on the profile of me that exists on all of my subject guides – “Make an appointment.”  Students click on the link and see times when I am available (because MeetMe connects with my Google Calendar) and can request a time that works for us both.  I get an email from Doodle, click on a link to confirm (or reject) the appointment, and the meeting is automatically added to my calendar.

I liked how this worked.  It was easy and convenient. But it isn’t perfect.

For years, students at my institution have been able to fill out an online form to request an appointment with a librarian. Students give us a bit of information about the project, their topic and their availability. The form is sent to all of the librarians, and the most appropriate (or most available) librarian “claims” the request and responds to the student. This is great for record keeping purposes (info is automatically entered into a database) and wonderful for students who don’t know who they want to meet with, but it can waste time if the student already knows who they need to talk to.

So I’ll keep the “Make an appointment” link on my profile and see if there are other ways to use this service.

Library publishing services: share what you’re up to

I should have posted this before, since the deadline for submissions is tomorrow, but my library is working with the Monroe County Library System to put together a Library Publishing Toolkit.

A working printing press in one of the labs at the FHTW in Oberschöneweide. CC image courtesy of flickr user tölvakonu

Proposals are due tomorrow (February 15, 2013), but all we are asking for right now is a 300-500 word abstract about the services and strategies that your library. 24 hours is plenty of time to write 300 words!

If you are doing some interesting or innovative things related to publishing, think about sending in a proposal. Some questions we are hoping to answer include:

  • What programs and services are offered by libraries to writers?
  • Does your library help users develop curated content to publish either in print or digital form?
  • What strategies are being used to select items for digitization?
  • Has your library identified unique print materials to be digitized and potentially sold?
  • Has your library developed partnerships with other agencies to support digital publishing?

If your abstract is selected (and you’ll know pretty quick), the final brief papers (just 2-5 pages) will be due on April 22nd.

See the complete call for papers on the Library Publishing Toolkit website. Submissions are accepted via email to browna@geneseo.edu.

Explaining science using simple words: Up Goer Five

While thoroughly enjoying the recent #overlyhonestmethods meme on twitter I came across the #upgoerfive meme.

This latest meme was inspired by an xkcd comic that attempted to explain the Saturn V rocket using only the 1000 most common English words.  Saturn V becomes Up Goer Five.

So Theo Sanderson created a text editor that only allows you to use the 100 most common English words and challenged scientists to explain what they do using simple language. It isn’t easy.  Anne Jefferson and Chris Rowan (of the excellent blog Highly Allochthonous) then created a tumblr blog collecting examples, like this description from volcanologist Lockwood Dewitt:

I like rocks. Also high places that sometimes act like they’re on fire. But what I really like is sharing what I know about rocks and high places, and how those things came to be. What made them? What moved them? Why are they the way they are? I think the answers to these questions are important, and I think people should know more about them. So I use words and pictures to show everyone how beautiful and amazing rocks and high places are, why they’re important to us, and why it’s important to know about them. Sometimes I even get to take people to see rocks in real life, which is the best part of what I do.

I tried my hand at explaining my job in simple language:

I help people learn about the different types of stuff they can find on the computer. I help them find books and computer stuff they need to learn about the world around them.  And I help them learn about how people tell other people about what they learned.

One of the first things I thought of was how this forces you to really think about the topic you are writing about, because you can’t rely on the jargon you normally use.

My next thought was that this could be a useful exercise for students, and could help them understand the concept of “putting something into their own words,” a concept that I talk about often in plagiarism workshops.

The first part of putting something into your own words is to really understand what you are trying to say, a step that students sometimes skip when putting together their term paper at 1am the morning before it is due.

So this might be an interesting challenge for students: ask them to use the Up Goer Five text editor to explain their research or term paper topic.

claimtoken-511bdf0b89fa2

Follow

Get every new post delivered to your Inbox.

Join 83 other followers