Whether a library is designing a building or a program, the first premise of designing for impact is figuring out what impact you’re trying to make and how you’re going to assess whether that impact is occurring. One of the most common buzzwords in librarianship today is “outcomes, not outputs.” In other words, measuring not quantitative metrics of what libraries do, such as circulation or visits, but what impact those activities have on the lives of their patrons.

However, determining how to measure outcomes can be a challenge. To help its grantees, and by extension other libraries, measure their impact in an effective and comparable way, the Bill & Melinda Gates Foundation’s Global Libraries initiative has come up with a set of common indicators and a methodology for measuring them. LJ caught up with Jeremy Paley, associate program officer of Global Libraries, to find out more.

“We know it’s a big change from the historical way of measuring,” Paley says. Paley advises libraries to start by measuring outcomes of one program rather than all ­offerings.

“At the most fundamental level, it’s about surveying your users,” Paley says. “A lot of libraries are already surveying those users; it could be about changing the questions to be about outcomes and perceptions of changes in lives.”

To design an appropriate survey, “it’s a question of examining your service and asking, ‘What is its goal?’ Sometimes you have to peel back the layers of history to get at that goal,” Paley says.

Driving design

To use outcomes measurement to drive design, it is often necessary to survey more than once and to look at answers in combination—as well, of course, as asking the right questions.

For example, Paley tells LJ, “in our grant in Lithuania there was an example of design for impact that impressed me,” he recalls. The library was doing extensive training for staff on new information technologies with the goal that the staff would in turn train users. But while staff skills were on the rise, user skills weren’t changing. The library realized it needed to provide training on how to teach as well as on the content itself. “Then they saw the needle move,” Paley explains. “Measuring these things can make a big impact on how you design your program as long as you’re measuring the right indicator linked to the outcome you’re trying to achieve.”

Once a library has gathered the data, Paley says, there will often be multiple possible responses. But it doesn’t matter which the library tries first. “You just kind of do something, and then it’s about measuring iteratively,” Paley says. “You don’t see the measurement as a one-time thing.”

A tool for IMPACT

For its own grantees, the Gates Foundation has developed a sample of 41 indicators (plus another 53 optional ones) that “have been vetted by people around the world as generally meaningful for advocacy and for adjusting the program.” They focus on seven common issue areas—digital inclusion, culture and leisure, education, communication, economic development, health, and government and governance—and translate to just 20 survey ­questions.

“We’ve made a compromise to make it possible to adopt a system like this,” Paley explains. The indicators look for contribution, not attribution. In other words, they measure whether the library is one of the causes of improvements in the lives of library users, not whether (or how much) the library is directly or solely responsible.

The foundation’s indicators are not only available to its grantees. Global Libraries has worked with the University of Washington to develop them into the IMPACT survey tool, Chris Jowaisas, Senior program officer at the foundation, tells LJ. The survey is open in beta mode and available for free to public libraries at impactsurvey.org. So far, about 100 libraries have signed up.