Purpose & Goals

While traditional metrics of student success commonly used by institutions focus on outcomes such as Grade Point Average, retention, and graduation rate, it is uncertain whether students’ own perceptions of success aligns with these institutional definitions. With a growing emphasis on student-centered assessment, the presenters from the federal funded research project will share the preliminary findings of undergraduate students’ academic success that were conducted at a public research university. We will also share the process of coding open-ended responses, which is crucial for ensuring inter-rater reliability of qualitative data. Lastly, we will explore ways to expand on the meaning of success and what implications might this have for our practice.

Design & Methodology

This project reflects the development of survey instruments created as part of an Institute of Museum and Library Services (IMLS) grant funded research project. The poster will focus on analyzing qualitative data using thematic analysis of an open-ended question on the survey instruments (Survey 1-Academic Engagement and Success Survey and Survey 2- Online weekly journaling for 8 weeks). After collecting student responses in Survey 1 to the question “What does academic success look like to you?” during the pilot phase, the research team collaboratively analyzed the qualitative data. This process aimed to enhance inter-rate reliability among reviewers and validate the accuracy of the findings. Guided by the Principal Investigator (PI) who underwent training in the educational psychology and adapted Tesch’s Eight Steps in the coding process, the team also facilitated additional discussions to ensure consensus in defining the meanings derived from the data. This coding process was iteratively repeated until all members of the research team members reached a consensus. This rigorous procedure was essential, as the open-ended responses would be later be converted into multiple choice format during the implementation stage at other universities. Reference: Tesch.R. (1990). Qualitative research: analysis types and software tools. New York; Falmer.

Finding

The final codes revealed that students define success by both traditional measures focusing on tangible results and achievements such as getting higher grades, and process-oriented theme emphasizing their personal growth and social enrichment such as working hard. The frequency of the codes revealed that students’ most frequent definitions of success involved getting good grades, gaining knowledge, achieving goals, putting in effort, and staying organized. Additional codes revealed the full picture of how student participants conceptualize success, such as helping others and use of interpersonal skills.

Action & Impact

The codes developed from student responses were used in a secondary instrument, an 8-week journaling activity. This allowed us to determine whether their definitions of academic success from the earlier survey align with the findings of reported through the online weekly journals. Also, this definition was also used to estimate how well they met their own definition of success at the end of the 8-week period. These findings were compared with other collected data, such as frequency and type of academic engagement, to better understand student behavior and its impact on their success. These instruments are being repeated at two separate institutions following the pilot study.

Practical Implications & Value

The library assessment community will be able to understand the process of low-tech thematic coding of open-ended responses so that they can apply the method to their own assessment projects. Additionally, they will be able to better understand the survey instruments, which will be shared widely via a project website at the completion of the research study in August of 2024. The community will be able to better use and adapt the instruments to their own populations to assess the impact of academic engagement (including library use) and psychological factors (such as motivation) on both traditional and nontraditional measures of student success.

View Poster (PDF)