Purpose & Goals

Both the length and frequency of surveys can increase survey fatigue and reduce response rates. One way to address non-response for long surveys is the use of matrix sampling method, in which only a segment of questions are directed to a potential survey participant. By spreading the distribution of questions randomly to participants, the pool of responses broadens. This method has been successfully applied in large, national surveys such as the National Health and Nutrition Examination Survey (NHANES) and LibQual+ Lite, increasing response rates without impacting validity of responses (Thomas et al., 2006; Thompson et al., 2009a, 2009b). Thomas, N., Raghunathan, T. E., Schanker, N., Katzoff, M. J., & Johnson, C. L. (2006). An Evaluation of Matrix Sampling Methods Using Data from the National Health and Nutrition Examination Survey. Survey Methodology, 32(2), 217-231. Thompson, B., Kyrillidou, M., & Cook, C. (2009a). Equating scores on “Lite” and long library user survey forms. Performance Measurement and Metrics, 10(3), 212–219. https://doi.org/10.1108/14678040911014202. Thompson, B., Kyrillidou, M., & Cook, C. (2009b). Item sampling in service quality assessment surveys to improve response rates and reduce respondent burden. Performance Measurement and Metrics, 10(1), 6-16. https://doi.org/10.1108/14678040910949657.

Design & Methodology

We will discuss our application of this method to extending the number of different surveys presented to the same sample population using the same survey protocol, increasing our reach without additional work of running multiple survey structures. We conducted two separate survey protocols - one approaching people in-person (Food for Thought), the other a click-through survey based on the MINES for Libraries® protocol (MINES Re-boot). For the first year of the Food for Thought (aka “Donut Survey”), we kept it simple. After informed consent and demographics questions, the system presented one of three surveys selected randomly. In the second year, we added screening questions regarding frequency of usage of library resources. Those who indicated that they never or rarely used the library were directed to one survey asking more probing questions. Of those who indicated that they used the library resources at least occasionally, the system then presented one of three surveys randomly. For the MINES Re-boot, the survey system randomly presented one of three different surveys to our respondents, each set of questions addressing different issues or concerns. Demographics questions were in a separate block outside of the randomization. Because each of these protocols has provided strong results (over 1,000 responses each year), we were able to collect enough data to conduct subgroup analyses for most demographic groupings. Thus, we were effectively able to run three different surveys using just one protocol.

Findings

From the first year of the Food for Thought (in-person) survey, we had 1,095 valid responses in FY2022, and the responses were evenly distributed across the three survey branches. For the second year, there were 1,241 valid responses, and of these, 72 were non-users. Of the remaining 1,169 responses from users, the survey branches were evenly distributed. For the MINES for Libraries protocol, which only started in January 2024, the responses have been less evenly distributed, suggesting that there may be issues with the questions of two of the surveys that are resulting in non-responses. This will be analyzed and addressed in our poster.
Action & Impact    The results of the Food for Thought surveys were incorporated in different evaluations. For example, we included results of questions regarding the use and need of foreign language materials with our comprehensive and holistic evaluation of the library's collection supporting the learning of non-English languages and cultures.

Action & Impact

The results of the Food for Thought surveys were incorporated in different evaluations. For example, we included results of questions regarding the use and need of foreign language materials with our comprehensive and holistic evaluation of the library's collection supporting the learning of non-English languages and cultures.

Practical Implications & Value

The success of the matrix sampling method with the Food for Thought survey led us to apply the same method to the MINES for Libraries re-boot. Thus, we have gathered input on our metadata and the findability of our resources without the additional burden of implementing separate surveys. We are able to create reports from this one data set quickly and easily, without the overhead of separate reports.