Purpose & Goals

In Spring 2022, the University of Alberta Library (UAL) Assessment and Insight Team (AIT) inventoried the services and activities of the various units of UAL in order to identify gaps in assessment activities and to determine where AIT might provide assistance. We discovered that assessment practices were significantly variable regarding teaching and consulting activities of UAL's librarian community. As teaching and consultation form a substantial portion of the work for much of UAL's librarian community, the AIT team planned to develop logic models for teaching and consultation, (from the teaching logic model exemplar in the Canadian Association for Research Libraries Library Impact Framework, released in 2021). However, the leads for this project (the authors) soon realized that there was a conflation of practices and ideas related to consultation and instruction. They undertook an assessment of current consultation practices, with the goal of delineating the two functions and also exploring the needs of the librarian community regarding the assessment of consultations. At this stage, we were not focused on the impact of consultations from the user perspective. Our assessment activity was intended to be informal in nature and initially for the purpose of developing a logic model, which would include assessment tools for the UAL librarian community. However, based on the results from the overall assessment project, the goals of the project evolved. In the end, our question changed from "How do we conduct and assess consultation" to "How do we support consultation practices?".

Design & Methodology

Our literature review at the start of this project found that generally there were few assessment methods among Canadian academic libraries for research consultations and that there are fewer formal assessments of one-on-one consultations compared to group instruction. Literature that we did find focused on measuring the impact of research consultations from the user perspective, which was not our focus. There was consensus that research consultations are important to support student learning, but there was little to no literature on how to conduct effective research consultations. To learn more about current assessment and consultation practices at UAL, we developed a survey that asked broadly about both. We used Qualtrics to host and distribute the anonymous survey. It was sent out in July, 2022 via a Google Group email to all UAL staff, librarians, and archivists, and remained open for one month. Respondents were asked to provide their email address if they wished to participate in a followup focus group to take place in fall 2022. The survey had 25 responses overall, which was a 15% response rate. However, although we sent it to all library staff to capture all consultation activity, from the responses we received, it was evident that most of the respondents were of the librarian community, which would indicate a much higher response rate from the main target audience (~37%). Of the respondents, 14 volunteered to participate in a focus group to further discuss their consultation practices and assessment needs. These volunteers were split into two groups to facilitate greater conversational flow. The focus groups were facilitated by the authors using a hybrid meeting format since many volunteers were still working from home at this time. The focus groups consisted of engagement and discussion type questions.

Findings

Our survey showed that consultations are highly variable in terms of purpose, audience and process, though some generalities did emerge. The most common reasons for consultations were related to coursework or research. Although most respondents described a similar approach to their consultations, practices varied depending on the consultation purpose or audience. Regarding assessment, many respondents felt that they lacked the time to meaningfully assess their consultations and that user feedback surveys distributed after a consultation could be a burden for patrons to complete. However, respondents stressed that knowing they were being effective during their consultations was important. Self-evaluation was identified as the most common form of assessment, but many respondents felt they lacked the time to thoughtfully review the results and implement changes. After reviewing the survey data, our question changed from "How do we conduct and assess consultation" to "How do we support consultation practices", which guided our focus group discussions. At the focus groups, participants identified several reasons for attending: supporting sustainable and effective consultations, skills development, and knowledge sharing. Participants noted differences between consultation and teaching, reinforcing that consultation practices are highly variable and individual. They also recognized that their experience of a successful consultation might be different from their patrons, and that overall it was important for everyone to have a good experience. Concerns of the efficacy of consultations continued to come forward, as well as the sustainability and quality of this service. However, it was stressed that consultations should not be evaluated simply for the sake of it. Given that extra time is always in short supply, any evaluation should be meaningful and generate positive changes in librarian's skill sets and student experience.

Action & Impact

Based on our findings, we developed a consultation toolkit that contains a mix of supportive and self-assessment resources, and includes multiple options to suit the needs of highly variable and individual consultation practices. There are three sections within the toolkit: 1) general best practices for conducting consultations; 2) training resources; 3) strategies for improving consultation practices. The third section is broken down into five sub-sections:

• Recognizing Student Learning and Cognitive Load
• Asking Effective Questions
• Conducting Effective Self-assessment
• Collecting User Feedback
• Turning Feedback into Actions (for Improved Service Delivery)

Use of the toolkit is optional, and assessment tools are intended for the purpose of personal reflection and improvement of consultation effectiveness. The data generated by these tools is for librarians' personal use and professional development only, and is not intended to inform librarian performance reviews or system-wide pracitises. However, librarians may choose to report on their evaluation/reflection activities and any related professional growth, and/or identify professional development needs. In addition to improved consultation practices, we hope that this toolkit will help to foster a greater culture of assessment which is still developing at our institution. It was also suggested that a community of practice (CoP) dedicated to consultations could be established. This recommendation has yet to be implemented due to ongoing discussions about which library unit should be responsible for developing and maintaining the consultation CoP.

Practical Implications & Value

This is an example of a multifaceted assessment project that engaged a large portion of the professional librarian community at a single institution. This assessment was unique in that we were assessing a library service—research consultations—from the perspective of the population delivering the service (in this case, primarily librarians) rather than the users of the service. There was initial concern that librarians might feel defensive about being "assessed" on their practices, and that there could be a lack of participation in this project. However, effectively framing the assessment project as a supportive endeavour, rather than as "peer judgement" of librarians' delivery of research consultations, demonstrates how a project like this can be a success. This assessment project evolved organically and responsively to the data we collected. We believe this is an important element to consider when planning an assessment project as direction can and should change as the project evolves. This project has led to evidence-informed decision-making to support the development of consultation practices. Further, the data collected has led to the development of a tangible, practical resource that can support librarians as they continue to deliver the important service of research consultations.

Draft Paper (PDF)
View Slides (PDF)