Purpose & Goals

Assessing a service provided by a library consortium adds an extra layer of complication to any assessment project. Scholars Portal, the service arm of the Ontario Council of University Libraries, developed a framework to guide assessment projects for its services. This paper describes the first implementation of this framework. It was used to identify strengths, opportunities, and next steps for the alternate text format service, the Accessible Content ePortal (ACE). The decision to choose ACE as the first candidate for the assessment framework resulted from recent changes to the service and evolving accessibility legislation and standards.

Design & Methodology

The assessment framework builds on a previous Library Assessment Conference presentation from Scholars Portal, “Evaluating from Arm’s Length: Assessing Services Provided by a Library Consortium” (Pagotto, Barrett, and Pereyaslavska, 2016) which recommends taking a comprehensive view of consortial services, including stakeholder and community perspectives, and using both qualitative and quantitative methods. Within this framework, three methodologies were selected to assess the ACE service: a comparative analysis of similar services, a quality assurance review, and a survey of staff stakeholders.

Findings

This assessment project has enabled us to articulate the value of the ACE service, understand its place in the Canadian and International alternate format service landscape, and identify specific steps that would improve the quality of the service and reduce barriers to its usage. As the first project created within the Scholars Portal service assessment framework, it has also been an important vehicle to test that framework and its methodologies.

Action & Impact

Recommendations about enhancements that can be developed for the ACE service are currently being reviewed by the consortial governance bodies. Specific, actionable recommendations about the most effective ways to implement or manage the service locally have also been distributed to the local ACE coordinators at each consortial member library. Internally, Scholars Portal is working to revise the service assessment framework based on what we have learned from this first project. The framework, along with the findings of this assessment, will be shared publicly with our community.

Practical Implications & Value

Services provided at the consortial level are complex, involving a variety of different stakeholders and different institutional priorities across member libraries. The ACE service assessment project responds to both shared and individual library requirements to provide accessibility services to users, faculty and staff. By looking at both local and consortial needs, this project helps to validate the new service assessment framework. For Scholars Portal, introducing the service assessment framework represents a step forward in our plan to periodically assess our services comprehensively, transparently, and appropriately.