Purpose & Goals

Artificial Intelligence (AI) refers to the development of computer systems and software that can perform tasks that would typically require human intelligence. These tasks may include problem-solving, learning, understanding natural language, recognizing patterns, perception, and decision-making. All of these tasks are critical skills and abilities for Library Assessment. This paper will discuss and frame the issues around the use of AI for library assessment purposes and the types of knowledge and training that is needed for assessment librarians to utilize AI tools responsibly and ethically. Our collective knowledge of AI tools in the library ecosystem is a step behind the people who develop these tools and step ahead of the rest of our society. Based on our personal experiences we have seen that AI tools are used in libraries to facilitate and augment our information retrieval skills and educate our patrons on the responsible use of these tools. Little systematic work has taken place so far to understand how library assessment operations can support these activities. Over the past year, our research has been deeply immersed in understanding the multifaceted implications of AI for libraries. This includes an analysis of AI tool utilization within academic libraries, evaluating AI literacy levels, and determining the requisite training and comfort levels necessary for effective implementation. Our investigations are now extending towards evaluating AI's integration within public libraries and state library agencies, signaling a broader applicability of our findings.

Design & Methodology

Our methodology is fundamentally exploratory, designed to dissect and understand the nuances of AI application in a library setting. We leverage a rich tapestry of data, literature, and firsthand professional experiences to construct a comprehensive overview of current practices and potential future directions. One of the co-authors, Leo Lo, brings a unique perspective through his development of courses on prompt engineering for Sage Campus and the Medical Library Association (MLA). His work, focusing on the nuanced crafting of prompts to elicit desired responses from AI models, adds a critical dimension to our analysis. Furthermore, Lo's extensive presentations across various forums on AI's societal impacts enrich our discourse, providing a grounded understanding of the broader implications of our work. In this paper, we draw upon these experiences and insights to frame critical inquiries about the integration and impact of AI tools in library assessments. We aim to engage our audience in a dynamic, collaborative discussion on key questions such as:

• Strategies for preparing libraries for AI integration and the specific role of assessment librarians in this process.
• Identifying the training needs for library staff and assessment librarians to competently navigate AI technologies.
• Examining the incorporation of AI tools within information literacy instruction and the systematic sharing of assessment outcomes related to AI tool usage within libraries.
• Evaluating library literacy programs through the lens of AI application.
• Establishing a supportive and proactive community of practice for AI in libraries.
• Defining the contribution of assessment to enhancing media literacy in the age of AI.

Through this multifaceted approach, our paper seeks to stimulate a forward-looking dialogue on shaping a future where AI tools are harnessed responsibly and innovatively to advance library services and assessment practices.

Conclusions

Our research and analysis have led us to identify a pronounced necessity for enhanced training focused on Artificial Intelligence (AI) within the library sector. This need extends even more broadly to society at large, emphasizing the urgent requirement for comprehensive education on AI issues. Our conviction is that library assessment occupies a pivotal position in this educational landscape. It has the potential to significantly contribute to ensuring that engagement activities centered on AI awareness and exploration not only meet but exceed learning outcomes and objectives. Through such targeted assessment, libraries can amplify their role as pivotal educational and informational resources, delivering value to their communities in ways that are both measurable and profoundly impactful. Moreover, this investigation has prompted us to reflect on several additional questions that could shape future research directions and practical applications within the field:

• How can library assessment methodologies be adapted or expanded to more effectively measure the impact of AI literacy programs within communities?
• What best practices can be established for libraries to navigate the challenges and opportunities presented by AI, ensuring ethical use and accessibility for all patrons?
• In what ways can libraries serve as community hubs for AI education, fostering a culture of informed and critical engagement with AI technologies?
• How can the success of AI-related engagement activities be quantitatively and qualitatively assessed to demonstrate tangible benefits to library stakeholders and communities?

These questions underscore the ongoing journey of discovery and adaptation as libraries confront the realities of an AI-integrated world. Our work suggests that by embracing their educational and assessment roles, libraries can lead the way in cultivating a society that is not only proficient in AI but also critical and ethical in its application.

Implications & Value

Our ambition is for this work to serve as a catalyst for ongoing dialogue and collaboration within the library community and beyond. We foresee regular gatherings—be they virtual or in-person—where professionals from diverse backgrounds convene to exchange insights on the integration and utilization of AI tools within library services. These discussions will highlight innovative practices and critically evaluate learning outcomes resulting from AI engagements, with a keen focus on ethical considerations and the appropriate use of AI technologies in both professional settings and broader societal contexts. By fostering a proactive discourse on AI's role in library assessment and its broader implications, this work aims to contribute significantly to the evolving landscape of library science and information management. It seeks to bridge gaps in understanding and practice, enabling library professionals to navigate the complexities of AI with greater confidence and ethical rigor. In essence, our work strives to lay the groundwork for a more informed, ethical, and collaborative approach to leveraging AI in libraries and inform the library assessment practices.

Draft Paper
View Slides (PDF)