On October 24, the Association of College and Research Libraries (ACRL) announced the publication of its 2024 report for the Project Outcome for Academic Libraries (POAL) toolkit. Data in the report offers a snapshot of POAL’s use and impact in FY24, from September 1, 2023, to August 31 of this year. The report is available as a free download from the Project Outcome for Academic Libraries website.

Project Outcome is a free online toolkit that provides a range of surveys libraries can download and distribute to help measure and analyze data about their programs and services. Participating libraries are also offered resources and training support so they can apply and share the results. During the period covered by the report, 143 schools in 39 states created more than 1,800 surveys, collecting more than 40,000 responses.

The standardized surveys allow academic libraries to aggregate outcome data and analyze trends by service topic, program type, and over time. Letting them see how the outcomes of their programs and services compare across their institution, Carnegie Classification, and the nation. “It’s a really valuable tool,” Gina Parsons-Diamond, ACRL program manager for data and research, told LJ. “The more institutions use it, the more benchmarking data everyone gets, and the more valuable it is for the field.”

ADAPTING THE PUBLIC LIBRARY TOOLKIT

POAL was launched in April 2019, based on the Project Outcome toolkit developed by the Public Library Association (PLA) that allowed public libraries to measure four key patron outcomes—knowledge, confidence, application or behavior change, and awareness—in eight key service areas.

PLA’s toolkit was launched at the American Library Association’s Annual Conference in San Francisco in 2015, quickly gaining traction in public libraries—and academic libraries as well. In 2017, the ACRL Board of Directors approved the creation of an academic library adaptation of Project Outcome, appointing a task force to oversee the process. The final product was field tested in 2018 by 54 institutions in all Carnegie Classifications across the United States and was launched at the 2019 ACRL conference.

Like the public library toolkit, POAL offers simple patron surveys, an easy-to-use survey management tool to collect outcomes, custom reports and interactive data dashboards for analyzing the data, and resources for taking action using the results. The seven surveys measure the same four key outcomes as the public Project Outcome toolkit, but the survey topics align with the concerns of academic libraries: Digital and Special Collections, Instruction, Space, Events/Programs, Teaching Support, Library Technology, and Research.

Three types of survey are provided. Immediate surveys, designed to be administered immediately after a program or service concludes, measure patron-reported learning, and consist of six questions—four Likert-scale and two open-ended; libraries can add up to three custom questions. The outcomes measured can be qualitative or quantitative. Follow-up surveys are designed to be used four to eight weeks after a program or service is completed, to help libraries better understand if patron behavior has changed or benefited as a result. Outcome Measurement Guidelines, included in the toolkit resources, provide additional support for outcomes-focused data collection. These guidelines focus on four areas—developing outcome measures, alternative data collections methods, measuring outcome data over time, and working with partners—with an eye toward demonstrating long-term, collaborative impact.

Dashboards are also provided to allow institutions to visualize the data they collect. Librarians can use the results in a range of scenarios, from collecting feedback for course improvement to sharing with library leadership and other stakeholders to show the effectiveness of programs and make the case for institutional, state, and federal funding. The most widely used survey topic is Instruction, noted Parsons-Diamond, often involving first-year orientation or one-shot instruction tied to a course.

“Our current editorial board chair, Kate Langan, at Western Michigan University, noticed that in her library orientation classes, all the students were saying they really liked going out into the stacks and figuring out how to find physical books,” noted Parsons-Diamond. “She made sure to include that in all the orientations moving forward, so that the students really feel comfortable at the library and can use it to their best advantage.”

FIVE YEARS IN

Since the toolkit launched, 971 academic libraries worldwide have created surveys, collecting almost a quarter of a million response. In the last year alone, POAL survey creation rose seven percent, with a 15 percent increase in responses. In FY23, NSSE (National Survey of Student Engagement) High-Impact Practices were added as a taggable feature in the toolkit; 38 schools have used the feature and 3,332 responses were collected.

The data is used by ACRL as well, to help track trends at the national level and by Carnegie class. To mark the project’s five-year anniversary, the association published Assessment and Advocacy: Using Project Outcome for Academic Libraries, a collection of case studies generated by POAL, edited by Parsons-Diamond. The collected information, dating back to 2019, includes data from COVID-era shutdowns, when POAL pivoted along with libraries across the country. “We published some resources about how to measure virtual programs so that the tool could still be useful to librarians,” noted Parsons-Diamond, “even in that extreme situation.”

Taking stock at the five-year mark, she is pleased to see how broadly the data has been used. This ranges from large, overarching projects—two Florida schools with a high transfer rate from the two-year to the four-year institution that collaborated to build a strong foundation for students moving from one to the other—to furniture placement, such as a survey response noting that students didn’t like the more relaxing seating in study rooms, preferring chairs at long tables to spread their work out instead. “That is the work of an afternoon for the library to rearrange the furniture, and it makes such a big difference for the students,” she said. “And they might never have known that nobody likes that study room because it has bad chairs if they hadn’t used the tool.”