Article Note: On a history of reference assessment

Citation for the article:

Logan, Firouzeh F., "A Brief History of Reference Assessment: No Easy Solutions." The Reference Librarian 50.3 (July 2009): 225-233.

Read via Interlibrary Loan.



I am doing readings on assessment of reference services lately since our director wants me to look into the topic. A big reason for it is connection to accreditation, namely the accreditation agency wants stuff on assessment of services, so the hot potato eventually makes it to me. As I am doing all this reading and evaluation of what we do now, one of the questions in my mind is what exactly do we want to assess. I am still pondering that one. In the meantime, Logan's article was one of the items I read.

This article basically provides a historical look at assessment methods for reference. You get a sense of what has been tried over time, how the concept of assessment has changed and evolved, and a bit on pros and cons. If you want to get a sense of assessment methods out there that have been tried out, this is a good place to start.

Some notes:

• "The dilemma is what to actually count and how to count it? Does a hash mark really reflect the reference transaction? And does a statistic adequately represent the quality and value of reference?" (225). This is one of the questions I am struggling with at the moment. We do have a basic hash mark tally sheet at the desk; we started keeping daily statistics this year after I did some advocating for it. I am aware the method is not perfect, but we needed something better than just doing a sampling one week during the school year, usually close to the end of the semester.
  
• Logan discusses Samuel Green's 1876 piece in Library Journal where Green defines reference standards. Some of the things Green proposed are still with us today. I particularly like the line of "a librarian should be as unwilling to allow an inquirer to leave the library with his questions unanswered as a shop keeper is to have a customer go out of his store without making a purchase. . . ." (qtd. in 226). Basically this is a reminder to do our best and accept nothing less. And yet there are times when some staff do need to be reminded of something as simple as this. However, I would be wary of the comparison of a librarian to a shopkeeper. A library is not a retail establishment, no matter how often certain people wish to establish the image of libraries as businesses.
  
• The article also discusses the work done in obtrusive and unobtrusive observations, such as the 55% rule and Weech and Goldhor's. From the later, "it was also recognized that patrons seemed to care less about the accuracy of the help they received than the friendliness and helpfulness of the librarian" (228). For me, that confirms my experience where we may not always have the answer (we may have to refer someone elsewhere; we may not have a particular material or item available in our library, so on), but as long as you take care of the patron with friendly and professional service, the patron will be satisfied (or at least appeased). This type of study marks a shift in reference assessment from accuracy of the service to patron satisfaction. Patron satisfaction seems to be the dominant form of assessment now. Even though Logan claims that by the decade of 2000, we have moved to outcomes for assessment, I think a lot of libraries are still more focused on the satisfaction aspect. The surveys (for the library as well as campus services) we have done here mostly deal with satisfaction of patrons. For instance, reliance on tools like LibQual+, which mostly measure perceptions and satisfaction. I think a lot of lip service is paid to the idea of evaluating outcomes, but a lot still goes back to patron satisfaction and perceptions.
  
• "In the 2000s, a new vocabulary emerged. The emphasis in the literature was no longer on the numbers or even the quality of reference so much as it was on the role reference plays in instilling 'lifelong learning' skills and how reference contributes to 'information literacy'" (230). There is a move to the teaching or educational role of reference services. The question then becomes how do you measure that. Personally, this would be the kind of thing I would like to measure. Philosophically, I have always believed in the link between reference and instruction when it comes to helping our students.
  
• Here is the catch on patron satisfaction surveying: "Although patron satisfaction is an essential component of successful reference service and assessing satisfaction is useful, it has historically been proven that patrons are not good evaluators of quality" (230). This is probably the same reason why one should take student end of year evaluations of classes and professors with a big grain of salt. A lot of variables can be involved in these kinds of surveys including mood of the patron at the time. Logan goes on, "many people want a reference librarian to alleviate or confirm their uncertainties. Satisfaction surveys can measure whether the service was quick and whether the librarian was courteous and professional, but they do not seem to be able to measure the accuracy of the answer or the quality of the sources offered" (230). Yes, I think it is important to measure if your librarian or staff member is courteous and professional. However, one still has to look at the quality and accuracy of that service as well. I wonder if we, as a profession, actually put any stock in quality and accuracy, or if it is just all about satisfaction. It's the mentality of keeping patrons happy so they will keep coming no matter the cost. I am pondering standards of service at this point, measuring the quality of the service in terms of resources offered to patrons as well as professionalism in service.
  
• "Individual departments should develop their own list of qualities associated with good reference service. This list should include behavioral characteristics (i.e. attitude, ability to communicate, and approachability), basic knowledge of resources and collections, subject knowledge, and reference skills (the ability to discern appropriate level of help, when to refer, use of resources, time limitations, interviewing technique, relevance, accuracy, perspective, and bias)" (231). Note that it is crucial to tailor this to your specific library.
  
• "There is no ideal measurement tool, but every reference department must nonetheless examine its service, not because of danger of extinction, but to set proper departmental priorities and define and articulate its level of commitment to meeting people's information needs" (231).
  

In the end, I have to concur with Logan: there are no easy solutions. But if we are to learn, grow, and provide good service, we have to keep looking for those solutions.