Booknote: Collaborative Information Literacy Assessments

What follows is my short review of the book as I posted it on my GoodReads profile. I did take some notes out of the book of concepts and ideas I wanted to remember, and I am including some of those after the review. I will say that I often found reading this book a bit more useful than some of the sessions from the Library Assessment Conference I attended back in October. I will grant that some of the reason may just be my learning style: I am a textual learner, and I do like reading at my pace to reflect and figure things out. On the other hand, well, let's not digress further.


Collaborative Information Literacy Assessments by Thomas P. Mackey
My rating: 3 of 5 stars

What can I say? Like many of the LIS books that I read, this had some parts that I found more useful than others. For a book like this I am looking for the following: clear explanations of how things were done; if possible, samples of worksheets, instruments, guidelines, etc., and how well can something be replicated or adapted to my situation. Practicality over just theory is what I need. Overall, most of the pieces offered in this book were practical, although in some cases, they were a bit heavy on the stats and not enough on the how something was done part. Don't tell me "we got a team together, we talked about it, and poof, assessment happened." What did you talk about? What kind of questions did you ask? What instrument or product came out? Those are the things I honestly want to know. Also, at times, some of the situations and campuses seem on the ideal side. Apparently, faculty and librarians get along splendidly in these places. I am fortunate I have some receptive faculty here to get some things done, but I have been in places where that is way far from the norm. I do not know if that is just a shortcoming of the book or the profession at large. I mean, there is a lot in the LIS literature about the difficulties librarians encounter in trying to implement IL and work with faculty (solutions, well, that is so-so, but I don't think it is hopeless).

The part I found most useful for me was the chapter on student self-assessments. I like the idea of putting more of the learning in the student hands and making them have a stake in the process. I did photocopy that chapter for later reference. I also took some notes from other chapters that I will add to my blog for later reference as well.

Overall, it is a pretty good book. I think library folks in academic libraries seeking to learn about assessment will find some good things here, but I do not think this is the definitive book on the topic. Just one of others we should be reading if interested in this topic. As usual with this kind of book, you may not need to read it cover to cover. Find the parts that you need and read those. The book does contain pretty good introductions that give a sense of what specific chapters are about, so you can decide what parts to read in full, what to skim, and what to skip.


View all my reviews

# # # # # 

 Selected notes I took from the book: 

How the University of Auckland defines information literacy: 

". . . a way of thinking and being that encompasses identifying, accessing, evaluating, organising, and communicating information relevant in all learning environments and fields of endeavour" (29). 

Possible survey question (for students): "As a result of this degree, my mastery of information sources and methods of information gathering is improving" (46).

Some  lessons on faculty and librarians collaborating on integrating information literacy into the curriculum and classes. This is from chapter 3 in the book:

• "First, integrating library sessions with course content is essential. Students respond more positively and learn more when the collaborating faculty mutually reinforce course goals, and when information literacy course material is tangibly connected to other course material graded assignments" (80). 
• Second, partnerships between library and academic department faculty must go beyond separate development of course objectives and assignments. Students respond and learn more when all course materials and activities interrelate. In practice, this means the faculty involved should prepare syllabi and plan class sessions and assessment instruments jointly" (80). 
• Finally, assessment of information literacy skills among undergraduate college students must be as varied as the skills themselves. Instructors will receive valuable information through indicators of student perceptions about their research skills, student-reported measures of time spent on assignments and types of information resources accessed, and analysis of student papers" (81). 

From chapter 5, which deals with assessment for adult learners. (For me, reminder that this article and this other one may be relevant and of interest to this topic as well).  Why online learning is useful for adult learners:

• Helps them be more independent as learners. Provides conditions to be reflective and confident as learners. 
• Flexible learning approach highlights student role of taking responsibility for his own learning. 
• A flexible online program can make instructor aware that students are at different stages in terms of confidence and experience with technology. 

Conference Notes LAC 2012: Parallel Session VI (Institutional Data)

Parallel Session VI: Institutional Data
Date: October 31, 2012, 10:30am

(This is the last session I have notes for. After this, the material was just not as relevant to my needs or the needs of my smaller college).

I. "Mining Library and University Data to Understand User Populations and Behavior."

• (Out of the three presentations in this batch, this was the one that interested me and seemed to have ideas I can adapt in my setting. Reminder to self: talk to the IR folks back home). 
• This project was done at the University of Miami. 
• In lieu of surveys, try to use existing campus data and data mining. 
• Data mining uses large data sets or multiple sets to seek out patterns. Form partnerships with other campus organizations that use large data sets. 
• Sample questions/things to ask: 
• Does book use change over time? 
• Who is using the library? 
• You may need to secure IRB approval for some things. Collect data from registrar, library, HR, and student activities. 
• Clean the data by replacing university ID's with project ID's.
• Data was put into a "restricted" vault area for privacy concerns. 
• In seeing patterns  while data mining, you may get more questions. 
• For collection development, checked days between catalog date and date of last checkout for materials acquired in a year. 
• Correlated library use to GPA. (Note they have a turnstile that scans IDs at their library, so this may not be as easy or realistic to do in other places). For their turnstile data, students do scan their IDs to enter the library, which allowed them to match ID data to registrar data. 
• Use ILL data correlated to instruction, maybe via papers' analysis, to see what the students are borrowing. 
• From the Q&A: 
• Is GPA correlating problematic given issues of grade inflation? GPA is the prominent measure of student success (i.e. it may be problematic, but it is the measure we've got). 

Conference Notes LAC 2012: Parallel Session V (Teaching and Learning III)

Parallel Session V: Teaching and Learning III
Date: October 31, 2012

(A couple of these sessions boiled down to see this and see that, the "this and that" being either a book or some website, so these are going to be short notes. Personally, if you are going to tell me that, just give me the materials. I am pretty good with texts as a learner).

I. "Charting Success: Using Practical Measures to Assess Student Learning in a Peer2Peer Research Model."

• (I was very intrigued by this presentation in part due to the format. The presenter built it on a LibGuide. Sadly, I was unable to get the exact URL, and I can't find it anywhere online, so I am guessing the presenter did not make it public. A pity. I still like the idea, and it may be something I would not mind trying sometime if I ever present. Update note: See comments section, author came over and gave us the link. Yay!)
• The focus is on a program for writing courses, first year. They did three 90-minute library instruction sessions per course.
• Research mentor program. 
• The overview was done with an xtranormal video. (link to the tool).
• Program focused on the first three ACRL information literacy standards. 
• Cycle: 
• Pre/post test survey
• Fixed choice questions
• Open ended question
•  Then, Likert scale
• Attitudes
• Perceptions
• And then, Reflective response
• Impact on process
• Valuable lesson

  II. "The A-Team: Making a Plan Come Together Across Campus."

• (Reminder to myself: this presentation did include a handout. I have in with the notes and a PDF copy). 
• Work done at IUPUI. 
• Instructional teams (cohorts) are made up of a lead faculty member (from the departments or the University College), a student mentor, an academic advisor, and a librarian.  They work in learning communities. 
• Developed a needs assessment survey for librarians to determine needs of program and condition. 
• Developed focus groups with faculty, librarians, administration, and students to discuss information literacy. 
• See Analyzing Your Instruction Environment: A Workbook.
• See also work of IUPUI PRAC (Program Review and Assessment Committee) information.
• See website of NILOA (National Institutes for Learning Outcomes and Assessment). Check their transparency framework (you can see the link for it right on the navigation bar on top).

III. "Conversations with Students: Assessment Leads to Adjustment."

• Work done at University of Kansas. 
• Project interviewing a sample of graduate students on their research skills. (I was intrigued by this idea. I think it can be adapted for our use here, maybe with our capstone students). 
• IL instruction then delivered in group discussion format was concluded to be desirable. 
• A sample of 9 students in social sciences. 4 sessions were recorded, transcribed, and analyzed. 
• The exercise does allow for librarian self-evaluation.
• Implications: 
• Point of need IL instruction. 
• Format of IL instruction. Students more responsive in discussion setting versus just an orientation. 
• From the Q&A: 
• Online tutorials? Can be helpful after the fact rather than as initial instruction tool. Also can use LibGuides. 
• Teaching research as a conversation about process with students, so they can see the big picture. 
• Idea for grad students: Dissertation bootcamp, sponsored by a graduate school (note that University of South Florida, students pay for this, about $30 dollars I heard. Anyhow, I thought we could also adapt this to our needs, though we would try not to charge our students. Food for thought). 

Conference Notes LAC 2012: Parallel Session IV (Communicating Value)

Parallel Session IV: Communicating Value
Date: October 30, 2012, 1:30pm

I. "Telling the Story: Library Assessment for University Leadership."

• Presentation about how their library, UVa, prepared for a presentation to their administration for funding. 
• See article in http://diverseeducation.com/article/48610/.
• What makes a good financial model: 
• Enhances decision-making. 
• Transparency. 
• Fosters accountability. 
• Provides stable funding for institutional priorities. 
• Provides appropriate incentives. 
• The library is an area that adds value to an institution, even if does not generate revenue. 
• Shaping the story. Articulate your purpose and mission, and you need to present it externally: provost, administrators, and others outside the library. 
• Describe and understand your constituents. For example, faculty want access to resources. Students may be interested in more spaces. 
• Share your ambition. Talk about your proactive innovations and visions, and again, share. 
• Paint your financial picture. Spaces, services, collections (what you spend money on). Tie costs to your mission statement. 
• ARL libraries can use peer institution data. For others, try to find peer data for benchmarking. 
• But a lot of this is input data. Next, we need to discuss outcomes data. 
• From the Q&A: 
• Deans may not "understand" the library, but we do. Faculty does too. Get faculty to tell your story too. 

II. "Making the Case for Institutional Investment in Libraries: The Value of Evidence-Based Narratives."

• The narrative: what we want administration to know. 
• Our contribution to student and faculty success. 
• Our contribution to university mission visibility. 
•  Develop short, focused messages. 
• Use data to support your narrative. 
• Focus on what is important to the institution. 
• Build on the library's existing strengths. 
• Tell your story using evidence. 
• Enlist support of others in the university community. 

III. "The ICOLC Balanced Scorecard Pilot.: The Value of Collaborative Parallel Play."

• This idea was originally for for-profit organizations, but it has been adapted to mission-driven non-profits. 
• Learn more about ICOLC at their site: http://icolc.net/.
• (Honestly, I did not really get a whole lot of much out of this presentation. All I remember is a bunch of charts). 

IV. "Consortia Value: The Orbis Cascade Alliance."

• See Megan Oakleaf's The Value of Academic Libraries. 
•  Recommendations: 
• Form an ongoing assessment group. 
• Embed indicators of success in all future planning efforts and initiatives. 
• Develop common data set. 
• Develop an assessment took kit. 
• Create a set of "elevator speeches," testimonials, so on. (I actually learned this one back at Immersion in Seattle summer a year ago). 
• (Overall, I get the impression I probably need to read the Oakleaf book. Of this session, the first two sessions were the ones I found a bit more useful in terms of some valuable nuggets). 

Conference Notes LAC 2012: Parallel Session III (Teaching and Learning II)

Parallel Session III: Teaching and Learning II
Date: October 30, 2012, 10:30am

I. "Project RAILS: Rubrics, Results, & Recommendations."

• See the RAILS website at http://railsontrack.info.
• The purpose of RAILS: A clearinghouse of information literacy rubrics. It investigates rubric reliability and validity. Another purpose is developing training materials (I will be looking through this site quite a bit in the coming days). 
• Case study from Belmont University, TN. They integrated rubrics into the general education curriculum. Taught at the freshman and junior level courses. 
• Case study from University of Washington-Bothel. Make rubrics scale.
• Identify artifacts that reveal actual information literacy learning. 
• A good rubric is detailed with analytic, precise, explicit performance descriptions. 
• Analytical rubrics are more effective than holistic ones. 
• Norming is critical for establishing a shared understanding of the rubric and achieving greater inter-rater reliability. A good rater is one able to work with others. 

II. "Impact of Library Instruction in Different Academic Disciplines: An Analysis of Student Transcripts and Course Syllabi."

• This presentation reported on work done at the University of Wyoming. 
• (This was probably the presentation I found most useful and intriguing of the set. I think there are some things here I can bring back to my library). 
• The question: does information literacy instruction make a difference? 
• See also article by Melissa Bowles-Terry  (the presenter) in EBLIP (Evidence Based Library and Information Practice) 7.1 (2012): 82-95. (Adding this article to my reading list). 
• See also the book Academically Adrift. (I have to add this to my reading list. I have been hearing a lot of buzz about it. I've seen reviews and rants about it too, so probably should read it myself to decide). In the book, check the findings relating to CLA tests for college majors. 
• Significance: 
• Data shows library instruction can make a difference in student success. 
• Students in under-served majors are not receiving library instruction. 
• Each under-served major has either an IL-related departmental learning outcome or an inquiry-based assignment. 
• You can collect GPAs and correlate them to majors and library instruction. 
• Research assignments + library instruction = Better student outcomes. 
• The idea is to make an information literacy audit (this is the idea that intrigues me. Need to read more on this). 

 III. "Rolling it up: the Evolution of an Information Literacy Assessment Plan."

• Report on work done at Keene State College.
• Started by administering the SAILS test due to calls for assessment. 
• Use it as benchmark rather than to compare your students from one year to the next.
• Moved to a new "tiered" assessment model in 2011. This then was reinforced by a second tier focusing on IL instruction. 
• Librarians were well-positioned for formative assessment. On a larger campus, feedback would likely be aggregated. (I think we can explore this at my library given our smaller population). 
• Tier 2: Informing IL Curriculum and Assessment: 
• Identify IL student learning outcomes (SLO) for every session. 
• Rubric assessment. 
• Classroom assessment techniques. 
• Data on use of CMS and LibGuides. 
• Data on faculty support. 
• Tracking information literacy outcomes at the reference desk. 
•  Some steps for one task force of librarians: 
• IL curriculum maps. 
• Identify department outcomes. 
• Identify instruction delivery models. 

 IV. "Assessment of Information Literacy as a Student Learning Outcome: Overcoming Barriers and Achieving Standards."

• Report on work done at Simmons College. 
• Analyzed 326 decennial accreditation self-studies. 
• 228 (69.9%) include IL in the document. 
• The majority place IL within undergraduate programs or general education. 
• Majority of instruction occurs at the course or class level. 
• Very little evidence of program integration. 
• In terms of assessment, 116 (35.6%) institutions assess IL as a student learning outcome. Most at the course or class level. 
• 23.6% used surveys (student perceptions mostly). 
• 21.5% used tests. 
• 14.7% use class/course evaluations (again, student perceptions). 
• Less than 1% assess IL through capstones or portfolios.
• Obstacles in integrating IL: 
• Resources, time, staff (a.k.a. the usual suspects)
• Faculty as access point. Buy-in is an issue. 
• Lack of consensus.
• On terminology. 
• On roles. 
• Student understandings and needs. 
• IL as an "orphan of the curriculum" due to disconnect between students, faculty, and librarians. 
• Differing cultures. 
• Lack of leadership for IL. 
• Talk of accreditation can help bring administrators on board. However, this motivates faculty less. For faculty, talk more about student learning, what they desire to achieve and about closing the gaps. 
• What librarians can do: 
• Tailor communication to your audience. For instance, "selling" to faculty: students will do well/better, meet learning outcomes, etc. 
• Implement an assessment plan. 
• We have existing standards with objectives and performance indicators (ACRL materials). Use them. 

Conference Notes LAC 2012: Plenary II (Vaidhyanathan Keynote)

Plenary II: Keynote.
Date: October 30, 2012, 9:00am
Speaker: Siva Vaidhyanathan
Topic: "Library Assessment and Big Data: The Need for Ethical, Legal, and Philosophical Analysis."

(As before, any personal impressions or additions I will put in parenthesis. Otherwise, these are my notes as I heard them)

• Speaker is author of the book The Anarchist in the Library. 
• Speaking of Google's mission to digitize all and make it accessible. But don't be fooled; they are a company. We can't let the organization of information and knowledge that we need be governed by one company. 
• Google gathers all it can in terms of data. It does it then targets advertising very well. 
• The Google book scan project. Initially, it was to sell digital copies. It went through the courts. However, the main goal was to gather text and data. Google is hoping to crack language codes and semantics, something which is very difficult to do (especially for computers). This helps it improve search, such as completing phrases based on what you type in the search box. It can answer question in English, so to speak. 
• There is no real account or history of how we got to big data. We are just here, as if it just happened. But it did not just happen. Policies were made. 
• Big data matters in: 
• Commercial. What Google does. What Target and Wal-Mart do. What the credit bureaus do. The data selling and buying. 
• National security, including law enforcement. FBI, NSA, etc. 
• Science and research. There is research money here. 
• Those gathering the big data have no incentive to stop or curb their practices. Not even any ethical discussions. 
• Two major motivating ideologies: 
• Market fundamentalism. Belief that the markets solve it all.
• Technofundamentalism. Making a better device than the one before will solve the problem (then rinse and repeat, so to speak).
• Another issue was the rise of consumer credit. This set patterns for companies to manipulate big data. This led to things like banks creating products such as derivatives, and we know the mess that turned out to be. 
• Surveillance is not just the government. Companies are doing it too: Facebook, Google, so on. 
• Allusion to the Panopticon idea. (self-reminder to also check notes from Doyle lecture on this topic at Berea College, which I have to post here sometime). We have invited this environment by democracy according to Foucault. The speaker says this notion, Foucault's, is wrong, but the surveillance is there. The speaker is not sure Foucault has the evidence.
• The speaker argues it is not an obvious panopticon. It is more a cryptopticon, where the extent of the surveillance and tracking is not really known or fully understood. 
• In terms of security, government security agencies are buying data from the market players and companies (hey, your tax dollars at work). Contribute to Facebook, for instance, and you are contributing to the NSA in all likelihood. 
• Problems with American surveillance: the false positives and the false negatives. For instance, the no-fly lists. A flaw used to be, supposedly changed by now that bad credit could get you in a no-fly list. 
• Film to watch: The Lives of Others.
• See Chris Anderson's article for Wired magazine, dated 6/23/08, on "The End of Theory: the Data Deluge Makes Scientific Data Obsolete." 
• Speaker describes article as arguing for the "googleization" of everything. A bit of hyperbole. 
• An error occurs in assuming data is everything. 
• The digital divide. 
• Summary: 
• Big, ethical, legal questions. Mistake is inventing things and asking questions later. In this case, we can avoid peril, and librarians can help (did not say a whole lot on how, but by this point, the lecture was getting to the end). 
• From the Q&A: 
• Selectivity criteria altered in big data. Mistake to think that with more data you lessen bias. 

# # # # # #
On a side note, Brian Mathews, The Ubiquitous Librarian, listened to this lecture, and he asks a question or two in his post "Are you the process or product"? 

Booknote: This is Not the End of the Book

This is my brief review of the book as I posted it to my GoodReads profile. In addition, I am adding some notes I took from the book, mostly quotes I want to remember, after the review. I was reading this in my few free moments during the Library Assessment Conference that took place this past October. I will say there are some passages in this book that were a lot more relevant and on topic than some of the lectures I listened to during the conference. But I digress.

The review:


This Is Not the End of the Book by Jean-Claude Carrière
My rating: 3 of 5 stars

The only reason I am not rating it higher is because the two authors-- Carriere and Eco-- do have moments when they just ramble on and on. After a while, you might want to skim some parts. But that aside, the book is set up as a conversation between the two authors moderated by Jean-Philippe de Tonnac, and it is worth reading. The two have great insights on all sorts of topics related to books, and even on some topics that may barely touch on books. They talk about books, the Internet, libraries (personal and institutional), censors, antiquities, privacy, etc. They cover a lot of ground in this book. I would say it is not a book to rush through. Take your time with it. Brew yourself some coffee or tea, and read a bit here and there. Book lovers definitely owe it to themselves to read this book. Librarians will probably enjoy it as well. If nothing else, the two authors do prove convincingly that the book is not going away any time soon, no matter what any naysayer predicts.

View all my reviews


My notes from the book. These are some quotes I wrote down to remember, with some comments from me here and there:

Maybe this is something that librarians can should help with, Jean-Claude Carrière writes, 

"What the Internet provides is gross information, with almost no sense of order or hierarchy, and with the sources unchecked. So each of us needs not only to check facts, but also to create meaning, by which I mean to organise and position our learning within an argument. But according to what criteria?" (81)

He also states:

"Learning is what we are burdened with, and which may not always be useful to us. Knowledge is the transformation of that learning into a life experience" (76-77).

Carrière on the notion of filtering and the literature we read in school, the idea that such may be too purified:

"This notion of filtering out naturally makes me think of wine that is filtered before drinking. You can now buy wine that is sold 'unfiltered.' It retains the impurities that sometimes lend a wine its particular flavour, and that are removed during the filtering process. Perhaps the literature we tasted at school had been too heavily filtered, and therefore lacked the spice of impurity" (106).

 Carrière on the reading of old books:

"Just as our journey through life, our personal experiences, the time in which we live, the knowledge we imbibe, everything, even our domestic problems, and children's misfortunes, all of it has an impact on our reading of old books" (159). 

The context for the next quote is that Carrière and Umberto Eco are discussing the concept of authorship and some controversies that have surrounded it. This statement seems very appropriate and relevant to the current political climate in the U.S., applicable to many on the Right wing of the political spectrum of the U.S. these days. Carrière states:

"But in the absurd world of conspiracy, gullibility has the edge over plausibility. Some people cannot accept the world as it is; being unable to change it, they feel obliged to rewrite it instead" (166). 

Carrière (most of the lines I liked came from him, can you tell?) now speaks on our knowledge of the past as it comes from books. This can be a sobering thought:

"A large part of what we  know of the past, which has usually come to us in books, is therefore the work of halfwits, fools, or people with a grudge. It's rather as if all traces of the past had disappeared, and our only tools to reconstruct them were the work of literary madmen" (179). 

Can you imagine if some later, more advanced (we hope) civilization tries to reconstruct the traces of our civilization if they got stuck with books by Ann Coulter, Michelle Malkin, and Rush Limbaugh?

Eco on the books we have not read. So yes, there are books you know about and can certainly talk about intelligently, even if you have not read them cover to cover. A good librarian can certainly do this, and I am sure the skill is also useful for things like reader's advisory:

 "There are more books in the world than hours in which to read them. And that doesn't just apply to all the books every published, but even only to the most important books of a particular culture. We are thus influenced by books we haven't read, that we haven't has the time to read" (269).

Eco goes further:

"So we can see that the world is full of books that we haven't read, but that we know pretty well. The question therefore is how we have come to know these books" (270). 

Eco goes on to give an answer. This is it, in brief:

"There are several ways to know something of books that we haven't read. Which is a good thing-- otherwise how would one ever find the time to read the same book four times?" (271)

Carrière on what a library, at least a personal library, should be. This was definitely one of my favorite passages in the book:

"It's important to clarify that a library is not necessarily made up of books that we've read, or even that we will eventually read. They should be books that we can read. Or that we may read. Even if we never do" (284).

Eco then adds:

"A library is an assurance of learning" (284). 

And finally, Carrière on what a book collection is:

"A book collection can be thought of as a gathering, a group of living friends, a collection of people. You can go to them whenever you feel a bit lonely or depressed. They are there for you. And sometimes I rummage through them and find hidden treasures I had forgotten existed" (327). 

 

Conference Notes LAC 2012: Parallel Session II (Teaching and Learning I)

Parallel Session II: Teaching and Learning I
Date: October 29, 2012, 3:30pm

(Note: This parallel session was to feature three panels/presenters, but one of them was cancelled due to the speakers being stopped by Tropical Storm Sandy. As usual, jotting down my notes from what I heard there with my comments in parenthesis).

I. "Library Faculty and Instructional Assessment: Creating a Culture of Assessment Through the High Performance Programming Model of Organizational Transformation."

• Opening question: Faculty culture is. . . . (we were to write a reply in a small card. Seemed like a nice idea. However, nothing was done with the cards afterwards. Seems like a bit of a lost opportunity, but still a nice idea). 
• Assessment motivations: 
• Assessment is hard. 
• A culture of assessment is harder. But this culture creates awareness. 
• Faculty status is often maligned. And yet there are some positive elements in faculty culture (I do admit that now that I have faculty status--albeit non-tenrure line-- my mixed feelings on the issue are raising questions for me, the reflective kind of question). 
•  Models (think of these three as forming the corners of a triangle): 
• Culture of assessment. 
• Faculty culture. 
• High Performance Programming. (see article: Nelson, L. and Burns, F. (1984) "High Performance Programming: a Framework for Transforming Organizations.' In J. Adams (Ed.) Transforming Work (pp.225-242). Alexandria, VA: Miles River Press. By the way, another case of having to dig up the citation because heaven forbid they provide it during the presentation. I really don't think this is that difficult for presenters to do, especially for something that seems crucial to the presentation).
• Culture of Assessment (for these, see also the 2004 issue of portal: Libraries and the Academy. Again, they did not specify which issue or article. However, based on the talk, I am betting it is it this article, "Creating a Culture of Assessment: A Catalyst for Organizational Change," that was featured in volume 4.3 of July 2004).
• Incorporated planning. 
• Leadership commitment.
• Staff responsibility. 
• Routine use of data. 
• Reward structure. 
• Training.  
• This is about knowing, being curious about student learning. 
• Takes time and effort, and administrators need to recognize that and support it-- training, education, data, analysis, etc. 
• Faculty culture. Note many librarians have it because they have faculty status (with whatever issues that brings). Look for positives and assets. What can we build upon as a strength. 
• Faculty culture: 
• Ambiguity.
• Autonomy. 
• Shared governance. 
• Individualism. 
• Rewards. 
• Idea: foster faculty learning communities. Through collaboration, create a culture of assessment. 
• Faculty Learning Communities (think of these four as a cycle): 
• Collaboration.
• Group cohesiveness. 
• Team-based vision. 
• Scholarship of assessment.
• Idea of something to do: 
• Build a learning community.
• Monthly instruction team meetings. 
• Focused on pedagogy, practical assessment work, and mutual support. 
• Annual instruction retreat. 
• Develop unit goals. Create a vision for instruction unit. 
• The monthly meetings build trust. Discuss what works or does not work. 
• "Leading change via commitment rather than administrative control in order to capitalize on values and characteristics of faculty culture." 
• Instruction assessment: 
• Professional development.
• Research focus. 
• Community of practice. 

II. "Collaborating with Campus Assessment Services to Evaluate Information Literacy Skills of Graduating Undergraduates."

• Case study from the University of Houston (main campus). 
• The rubric assessment only assesses what the student has done. 
• A process study with selected students: 
• A survey. 
• Video tape and interview, based on the survey. Also ask about research in their disciplines. 
• Conclusions then based on observations. 
• Skills assessed: 
• Process: 
• Search
• Identify
• Select
• Access
• Evaluate
• Product: 
• Selection
• Breadth
• Intergration
• Attribution
• Citations
• What can be accomplished: 
• Product assessment
• Learned information skill levels. 
• Establish benchmarks. 
• Create standard rubric. 
• Process assessment: 
• Establish a study procedure. 
• Gain insight on actual information-seeking behaviors. 
• Students listen and remember what faculty tell them. Thus, need to work with faculty on information literacy and assessment. 
• Library instruction program. Impact to seek: 
• Teach concepts over tools. 
• Expanded assessment. 

Conference Notes LAC 2012: Parallel Session I (Information Services)

Parallel Session I: Information Services
Date: October 29, 2012, 1:30p

(The parallel sessions featured three or four speakers. They usually had a track/common theme. Sadly, there often were papers/presentations in other tracks I wanted to hear, but they were in a different room. Since the timing was not consistently maintained--a paper could run over, or the order of speakers changed--running from one room to another was not always desirable. Oh well. At any rate, I will be jotting down notes for each presentation I listened to. As usual, my comments are in parentheses).

I. Topic: "Using a Mixed Method Approach to Assessing Roaming Services: A Case Study."

• This was the case study from Florida International University. They implemented a roving service for reference. (I will admit interest in this session as I think there are some ideas on roaming that we could adapt here at Berea with our reference students. The presentation did give me some things to think about). 
• Surveys revealed a preference for face-to-face assistance versus virtual options. 
• Any reference performed beyond the reference desk is roaming. 
• The program was funded by a student technology grant that allowed the purchase of iPads for participating librarians. (This is one option I am intrigued about since I do have the option of requesting an iPad in my workplace for instructional use. More on that later I hope). 
• They established guidelines of practice for the librarians. They devote 2 hours a week to roaming. They also participate in campus events as the library representative (a.k.a. as outreach. This is the kind of work I used to do at my former workplace, and no, they did not provide an iPad. It was not for lack of me asking for one. At my current place, outreach is an area we have barely touched, so things can get interesting). 
• You can learn more about the ipads at FIU at this link.
• The library literature has been very sparse on the topic of roaming. The presenters claim they found 3 academic and one public library program described. The articles found mostly discussed pilot programs. Most of the data from such articles were basic statistics, things like number of roamers, transactions, and other reference statistics. 
• They recorded 140 transactions from roaming at FIU for the Fall of 2012, as of the time of the presentation. Most questions do fall under directional, informational, and some tech issues categories. Transaction length varied from one to ten minutes mostly. 
• They implemented a small student survey of why they came to the library. They used the librarian's iPad to get students to complete it during roaming (the librarian, after providing an answer would get the student to answer the short survey on his/her iPad. I thought this was kind of neat, but I now wonder how much students might feel pressured to do the survey or free to decline doing it). Questions in the survey included: 
• How often do the students ask the librarian for assistance? 
• Satisfaction with the service. 
•  Would you be inclined to ask for assistance more often? 
• The location of the reference transaction. 
• The presenters said they also sought qualitative data. 
• They note that roaming is not a replacement for a reference desk. It is a tool for additional help for the students. 
• From the Q&A: 
• On using the iPad's camera: One idea is to do that "Day in the Library" activity, like ALA's Snapshot Day.  You can also use the iPad's camera for documenting space issues in your library building. (For the latter, you can probably do that with any camera, but I get we are showcasing iPads now). 
• Look to have a roamer and a reference person at the desk. (This is what got me thinking. We could have a librarian roam and the student worker at the reference desk, or viceversa). 
• How are the roamers identifiable? Roamers could have larnyards, name tags, maybe a tee shirt. 
• There was a question on how to contact a roamer if a student wanted to do this. (Not fully answered/settled). 

 
 II. "Shop Your Way to Service Excellence: Secret Shopping for Academic Libraries."

• This was the case study from UNC-Greensboro. 
• See also following article: Elizabeth Kocevar-Weidinger, et.al., "Why and how to mystery shop your reference desk." Reference Services Review 38.1: 28-43. (They just mentioned the issue and volume number. I assume this is the article in question after doing some searching. The whole journal issue deals with research post-Google, by the way. Again, would it kill people to provide a useful citation now and then?) 
• They decided to implement their mystery shopper program after their library did LibQual+ in 2008. 
• Initial outcomes: 
• Develop customer service values. 
• Conduct mystery shopping in the library. 
• Determine if training is needed on the basis of the mystery shops. 
• Repeat to see if the training had an impact on service. 
• Procedure: 
• A survey team was created. 
• The staff was told the mystery shops would happen, but not when they would happen. 
• The library got "shopped" in person, over the phone, and via online chat. 
• The focus was on the customer service experience. 
• Shoppers were trained. They were recruited from the UNC-G Department of Hospitality. 
• Anonymity was protected. Shoppers would not collect names of staff responding to a transaction. 
•  What was asssessed? 
• Greeting, follow-up, confirming satisfaction, appropriate referrals were done on a 1-3 scale where 1= poor, 2= satisfactory, 3= very good. 
• Respect, avoiding jargon, and "did we go the extra mile" were done on a yes/no scale. 
• The presenters argue that this process can provide better evidence than just a satisfaction survey. It can help enhance the "culture" of excellent customer service. 
•  Step delicately. Again, assure staff that this is not part of the performance evaluation. (In other words, you are not supposed to use or bring up mystery shops as part of annual performance reviews or such, tempting as I am sure some managers may find the idea). 
• You can also see their LibGuide for teaching customer service skills to student workers.
• From the Q&A: 
• You may have to check with your campus IRB before you try this on your campus. 
• On the question of a librarian teaching versus "just giving them the information/answer." The presenters were aware this could be an issue, but it was not a focus of the study. The sample questions for the shopping were made as generic as possible. 
• Note their library opens 24/7 (something that is unrealistic in a lot of places for various reasons), so effort was made by the shoppers to vary their hours.
• For incentives to the shoppers, there was some extra credit in one of the hospitality classes. They also got a small bag of goodies that included $10 in their student ID accounts. 

III. "Secret Shoppers in the Library"

• This is the case study from Georgia State University. 
• Often, reference services quality is assumed, but it is not measured. 
• Their "shopping" program was implemented to assess library employee reference skills. It sought to identify weak areas as a whole to inform employee training. 
• The project took place in 2010 for the library. In 2012 for reference services. 
• The evaluation form was based on RUSA guidelines with a rating scale and open-ended comments. 
• The students recruited for the mystery shopping came mostly from their LIS school. However, given the campus has a large, non-traditional population, a grad student in the library would not be unusual. 
• A result of the program was implementation of regularly scheduled reference interview training. There is an emphasis on reducing referrals. 
• Hiring for personalities (which a colleague and I could not help but say to ourselves, "wow, what a concept." Imagine that, hiring for things like personality and attitude, but I will not digress further). 

Conference Notes LAC 2012: Plenary I (Eaton Keynote)

Plenary I: Keynote.
Date: October 29, 2012, 10:45am
Speaker: Judith Eaton, President, Council for Higher Education Accreditation
Topic: "Higher Education in a New Era of Public Accountability: What Does This Mean For You?"

(As before, any personal impressions or additions I will put in parenthesis. Otherwise, these are my notes as I heard them)

• Some numbers: 
• There have been 200 new regulations since 2008 affecting accreditation and institutions. 
• 1 trillion dollars in student loan debt. 
• We are still awaiting the Higher Education Act reauthorization. 

• Some basics: 
• Dominant role of the government. 
• The compliance role for accreditation. 
•  A shift in who decides quality: from higher education to the government. 
• Diminished reliance on institutional leadership for academic quality. 
• Assuring quality in innovation: for-profits, MOOCs. We'll be tested on how these work.  

• Recall the Spellings Commission (Wikipedia article; archived homepage of the commission including reports and other materials): ongoing criticism of higher education in terms of credibility, rigor, integrity. 
• Higher education challenged: costs up, perceived value down.

• Expectations: 
• Accountability for evidence: student learning outcomes and institutional performance. 
• Accountability for transparency. 
• Accountability for jobs and earnings. 
• New accountability tools and sources of judgment: rankings, placement rates, graduation rates, comparability, foundations, the press. 
• Note that the comparisons, so on are not done by us in higher education. They are done by think tanks, foundations, so on (hmm, like the CHEA maybe?) so on. Foundations such as the Gates Foundation, by their donations, can help set policies (for good or ill, we have private foundations, which may or not really have the best interests of education at heart, so on putting in money, which thus helps set policies). 

• New era: Government and accreditation: 
• Partnership giving way to oversight. This is another shift. 
• Accountability only if you have regulation. 
• Everything is worth regulating, even to low details such as faculty having a biographical statement on a college website (I have actually seen this be a big deal in other places), general education offerings, so on. The speaker went on to question if any of this makes sense. 
• Government officials decide, not collegial/peer review. 
•  The drive for compliance drives out the improvement role of accreditation. 

• Why the concerns and the new era? 
• Money. $460 billion dollars higher education enterprise. 
• Price up and perceived value of higher education down. 
• Absence of jobs. Sure, there was a recession, but there is still an expectation of higher education contributing to economic development (which is somewhat true, but then again, are we basically saying all higher education is simply vocational education? It seems a lot of people would think so). 
• The presence of debt and defaults. 
• International competition, for instance the OECD (which the speaker right away rushes to "clarify" uses "questionable" methodologies. Sounds a bit more like the world is doing better than the U.S. in terms of education, and an American does not like it. The methodology may not be perfect, but that is the usual excuse in the U.S. when talks of why the world often does better surface. By the way, on the link, click then under topics to find the education data). 
• Another factor may be trust, or rather a lack of it in society at this moment (I can't imagine why). 

• What role for libraries:
• Help institutions know more about themselves. 
• Institutional performance. 
• Student learning outcomes: research, data, tools. 
• Help institutions with transparency. Provide evidence to the public. 
• Provide leadership for accountability. 
• Communicating ROI and value of your campus. 
• Participate in accreditation. 

•  Libraries have the tools and capacity in place: forums, assessment blogs, other opportunities. 
• We must lead to preserve the role of accreditation. Keep authority for academic quality within higher education. 

•  Summary: 
• Be a strong advocate for value of mission, peer and collegial review of quality. 
• Be aware of the accountability climate. 
• Sustain focus on library accountability. 

 

Conference notes LAC 2012: Plenary I (Simons Keynote)

Plenary I: Keynote.
Date: October 29, 2012, 10:45am
Speaker: John Simmons, Provost, University of Virginia (UVa)
Topic: "Dollars and Sense: the New Financial Realities of Higher Education."

(As before, any personal impressions or additions I will put in parenthesis. Otherwise, these are my notes as I heard them)

The three topics of the lecture:

1. the challenges facing public higher education. 
2. the University of Virginia finding faculty. 
3. what it means for libraries in general. 

(Simmons mentioned an October issue of The Economist for October 2012 he recently received shortly before this lecture. He did not give specific date, but based on the speech, I am guessing he refers to this special report on technology. I did check the TOC's for the print issues of the month, and none seemed to have anything on technology or higher education related to his speech).

Simmons referred to Time magazine's October 29, 2012 issue (vol. 180.18) on the topic of "Reinventing College."

Factoid: 1/4 of financial aid goes to for-profits, but those schools have the major dropout rates.

(Again, see Generation on a Tightrope for further reading on the topic of disconnect between college administrators and lay people).

• (Again. This is also consistent with Lombardi's keynote): The core of a good university is its faculty. Faculty feel they need access to all knowledge in their field. Research schools compete for the same pool of new faculty to hire. UVa authorized 57 faculty searches this year. 

• This means universities need to rethink strategic plans. Previous plans have been about where we've been and what we've done versus implementation. 
• Mentioned the book The Strategist. (as often happens in these lectures, speakers just toss out a title with little to no details. I am not going to comment further on the irony of a speaker in a librarian conference not providing a proper citation. Based on context and title, I am thinking he referred to Cohen's book here. If anyone knows different, feel free to comment). 
• The library comes in for the role of providing access to online resources (databases, etc.). UVa library has partnerships with their information services people via various laboratories. Also, there is assistance with digital literacy, which goes with online education.  However, online education is not a replacement. UVa sees technology as a way to deliver information, then use face to face time for application, so on (flipped classrooms maybe? Reminds me we had a presentation on just that topic a couple of weeks back. I have to find my notes on that). 
• Suggested for us to check out "recent" Washington Post articles on UVa. (No, he did not specify which ones. I am thinking this was a self-pat on the back).
• Suggested for us to read, or reread, Huxley's Brave New World.
• What this all means: 
• The library will always play a role. However, students and faculty may not realize how much they rely on the library: online access, digital labs, etc. Many initiatives are funded by the library.
• Access and preservation fall to the library. 
• What libraries need to do:  
• Align priorities with their institution's strategic goals. 
• Show faculty what you do. 
• Ask what can you let go to pursue new opportunities. (Implied is that you then let go whatever it is you identified).
• Focus on users and their needs. 
• Provide leadership and teaching opportunities (this, I would have loved to hear more about). 
• Collaborate with others on and off campus. 
• Students as "traditioned innovators." Librarians then have the role of enabling this: understanding of the past and how to bring it in to innovate. 

From the Q&A:

• How to get involved in "budget battles" (at least internally) and not neglect your constituencies? (this also comes from Lombardi's speech). 
• Librarians do need to have a voice with planners, budget people, so on. (Yes, that was about the extent of the answer. Kind of vague and general). 
• Need to correct the perception that faculty meetings are not congenial. Collaborations are often found around campus. The library needs to create opportunities so the provost and other administrators can engage with the library, so they can see how they do need to support the library and thus support campus scholarship. 
• Any administrator needs to know his/her landscape and know the areas of excellence for his campus (i.e. don't try to be an R-1 if that is not what you are). 

 

Conference notes LAC 2012: Plenary I (Lombardi Keynote)

Plenary I: Keynote.
Date: October 29, 2012, 9:30am session.
Speaker: Professor John V. Lombardi.
Topic: "Living in the Cloud: Who Owns It, Who Pays for It, Who Keeps it Safe and Will My Kids Inherit the Wind?"

(Any personal impressions or additions I will put in parenthesis. Otherwise, these are my notes as I heard them).

• The cloud is not quite there yet.
• Scholars and administrators look to the library more in terms of online resources, and they call on librarians more for support (from things like how to log in to a database from off campus to how to use a database). 
• Some guidance on this: 
• The old game is over. 
• Going alone is not really an option. There is need then for consortia and other forms of collaboration. 
• Initiatives to digitize everything need to be funded. Anything no digitized in ten years or so won't be real. (I thought that last remark was provocative, but I got the impression to most everyone in the room this was a given. Maybe I am thinking research in some cases may not be as thorough if scholars can't access certain things because they do not "exist," and the scholars then do not know how to find them. In this context I am thinking of a colleague of mine who is a music librarian, a field of endeavor where a lot of material is still in archives and where you do have to know where and how to look past Google.And that is just one example). 
• Librarians must engage in preservation. 
• Librarians and libraries need to keep taking on copyright issues. Ownership is crucial. (Personally, I've always said that we need a few people with cojones to actually go to court and challenge those who keep restricting copyright more and more. First sale doctrine should mean exactly that. However, I just don't see our profession being that kind of people. More often than not our profession prefers to make nice and "be civil.").
• Libraries must embrace and redefine special collections. 
• Libraries must develop the rhetoric to tell their stories. Lead in digital collaboration and access, keeping curation and access to unique collections. 
• Abandon the idea of the teaching function. This does not yield prestige. Move to spend digital money and get power. (This definitely irked me, and from the looks of some responses on the Twitter hashtag for the conference. I am just linking to the hashtag. Feel free to browse it. Anyhow, for me, it may well be that, in addition to the fact that I do believe in the mission of teaching libraries, that I am not looking for prestige. Then, there is the idea that even if you were to put it all online, someone still may have to teach people how to find it, use it, evaluate, so on. Our teaching function is not going away no matter what some guru says.). 
• Librarians need to be supporting commons, coffee shops, etc. (I am not even adding or commenting to this). 
• Participate in every single game. Be at decision points. (To an extent, it does sound like giving up on some principles in order to become IT gurus, but that is what he said. I do agree with the idea of being at decision points, but we can't be everywhere. Learning to choose battles, thinking in strategic terms may well serve us better than trying to cover everything.). 

Some notes from the Q&A:

•  For non-research universities, the primary challenger is other education providers, such as the for-profit schools. (This reminded me of the book I recently read, Generation on a Tightrope and whether one makes a stand on bricks or clicks. This is making a stand on clicks).
• Open Access (OA) and open publishing as solution? It is a great idea, and we'll see more. Professor Lombardi is not sure if it is a solution yet. There is a lack of a stable and effective model (I wonder what some OA advocates and experts would say to this). It is how quality is defined and knowing if you have quality or not, if you have a carefully calibrated competitive item. This is what it is about; it is not so much about access itself. OA has to "function within the guild." Also, "crowdsourcing is not going to work." We have to ensure a quality infrastructure.
• (Here is another doozy): Tenure is a 35-year investment on a small chunk of information, whether it is good or not. Investment in faculty members is the key in making a university good, NOT the library. 
• Skills to develop: 
• We need someone who can speak clearly to articulate what we do. Tell the story. What is the library's leverage point? (I don't think we are lacking people who can speak/write clearly to articulate what we do. There are plenty of high profile, articulate librarians who blog that you can go look up to for an example. Yes, I do mean some other blogs. Maybe as a profession we need to, again, think more strategically and collaborate more on telling the story, but it is not due to a lack of articulate librarians. Maybe in terms of presenters, and bloggers, we need more voices than the usual, already well-known voices. Just a thought). 
• Learn to mobilize your constituency. Note that faculty often complain individually. 
• Be able to talk the technotalk. It does not mean you become a technocrat (funny he says this now. See above. However, I do agree we do have to be able to speak the language. I mean, how else can we tell the tech guys what it is our library users need?). 
• Be able to write and present. When you write, people have to respond (this does assume you write publicly). 
• You always have to fight about the money. Money differentiates who is good or not on campus. Be there when people talk budget. (In other words, money does make the world go round. I do think it is a sad comment that it all boils down to money and who has it when it comes to who is "good" or not. By the way, "good" here may not mean what Dr. Lombardi and others think it means).

ARL Library Assessment Conference 2012: Initial notes and impressions

Last Friday was my first day back from attending the ARL Library Assessment Conference (LAC) in Charlottesville, Virginia. As I am taking some time to reflect and review my notes, some of which I will post here for my future reference, I find that I have some mixed feelings about the experience. There were some sessions that were interesting, but there were also one or two sessions that I found to be underwhelming. I will try not to say more publicly since one of the things I did learn at this conference is that you are only supposed to fawn on speakers. Say something negative, even in light jest, and they can actually get snippy (yes, this actually happened, and no, I am not discussing it further unless certain people bring it up).

So, what else did I learn right off the top? I found that smaller colleges and universities are woefully underrepresented. I guess I should not have been surprised by this. It is ARL after all. The reason I mention this is that a colleague of mine from another institution mentioned that the issue was brought up in a session she attended that I did not attend. It just made me think a bit. I also thought of this because in many cases the projects presented are done in places with bigger budgets, more resources, and more people, people who often do assessment as a full-time (or almost full-time) endeavor with little or no additional concerns. To folks like me, assessment is just one of the many things I have to do. My colleague wondered about that as well. For me, however, I don't think all is lost, so to speak. I have a good instruction team here, a supportive administration, some access to resources, and plenty of pluck. I did come back with some nuggets of wisdom I can adapt to our setting here.

I did hear from a few people wondering why PowerPoint slides, the presentation tool of choice, were not made available beforehand. Given that many presenters relied heavily on graphs and tables for their presentations, having the slides handy to take notes would have been helpful. I do not think it is unreasonable to request these so they can be made available beforehand. And by the way, this morning, as I was looking over my feed reader, I came across this post on "Ten Things I Did Not Learn in Library School, Academic Edition." The post has a few things I would disagree with, but that was caught my eye was this comment in the post by Jo that echoes something I have been saying for a while. She writes:

"Library schools need to be teaching educational psychology and pedagogy if we expect librarians to be respected and included in faculty circles! Just go to any library conference and listen to 15 minutes of the boring presenter-librarian reading their PowerPoints to the audience and you'll know we've missed that boat." 

 Yes, I sat through one or two of those PP presentations that were not exactly enthralling. I will grant this is not just LAC, but it is something to think about in our profession. 

At any rate, I am still sorting through my notes. I also jotted down some book titles; presenters often recommend books and articles for further reading, and I will add some of those to my notes as well. So, for the next few posts, it will just be my notes. 

On the side, I did manage to get some sightseeing in. I did get to see Monticello and, on the way back, Woodrow Wilson's birth home and presidential library. I can cross that one off my list of Presidential Libraries. I have a very small bucket list, but seeing the Presidential Libraries, official or not, is an item on that list. I will try to write down some of my impressions of those places later, probably over at The Itinerant Librarian since sightseeing is a more personal thing. And on a final note for now, I learned that driving through the West Virginia mountains to get to Virginia and then back home to Kentucky is one of the most scenic and beautiful experiences one can enjoy.