The E-Metrics Instructional System will help librarians collect and use data about networked environments Libraries today provide services and resources in a hybrid operating environment: there is the physical library and there is the electronic one. Because of this, libraries can no longer rely solely on traditional building-based measures of use. Managers and staff need solid numbers about all library collections and services to understand how patrons are using their progressively networked environments. They also need to present these new types of use data to funders and other community stakeholders. Now these numbers are within reach with e-metrics (statistics for network services and resources). Libraries may undergo short-term pain in collecting e-metrics, but the long-term gains will be substantial. Failure to capture and report these data leads to undercounting the actual use of libraries, frustrating the ability to demonstrate the true value of libraries to their communities. We now have a number of e-metrics at our disposal, but librarians must still grapple with deciding which ones to use - and how to apply them. The Information Use Management and Policy Institute at Florida State University is developing the E-Metrics Instructional System (EMIS), a web-based interactive instructional system to help librarians understand selected e-metrics, how to collect them, and how to use them for decision-making and communication. Funded through a National Leadership Grant from the Institute of Museum and Library Services, EMIS, to be completed by September 2004, will provide librarians with a freely accessible instructional system to learn about e-metrics, review definitions, and get help collecting e-metrics and using the data. Librarians will also find sample reports that are particularly useful for educating community-based stakeholders. Forms, worksheets, and other items for evaluating services and resources as well as links to other resources will be available.

Shifting numbers

Since libraries began connecting to the Internet in the early 1990s they have increased their investments in information technology, telecommunications, and the services and resources that take advantage of the newly acquired and/or upgraded infrastructure. Network-based services and resources offered by libraries today - digital reference, digital collections, online databases, e-journals, and ebooks - enable libraries to operate in an anytime/anywhere mode, providing patrons who have Internet connections access to content, services, and resources 365 days a year, 24 hours a day. Some libraries are experiencing stagnation in traditional services as libraries increase network services and resources (see "Statewide Public Library Visits and Reference Transaction Counts," below). Walk-in visits may be down or circulation statistics may be flat or declining - two key measures that libraries traditionally use to demonstrate their success.

The ups and downs of libraries

Some library governing bodies, community leaders, and others have suggested that decreases in usage should result in decreases in funding. While one would think that these plateaus and decreases would lead to a corresponding reduction in librarian workload, the authors have yet to encounter librarians or library managers who indicate that their workload is down. In fact, librarians and libraries are busier than ever. Other libraries are experiencing a rise in visits or circulation or both. But even if in-person usage and circulation remain steady or increase, it is clear that libraries are also being used differently through non-building-based library services and resources. The key question for managers is: What exactly is going on in libraries today? Without comprehensive national numbers, it's impossible to know with precision, but libraries are in a transitional phase. They continue to provide traditional services while simultaneously increasing network services and resources. These network services and resources are increasingly used by patrons. However, libraries, by and large, do not collect or report usage figures for network services. If they do, they keep them separated from traditional use measures, which can leave the false and dangerous impression that library use is waning.

The e-metrics menu

To incorporate measures of network service and resource use into ongoing measures of building-based service and resource use, librarians should know what measures exist, what they say about the library, and how to collect them. A number of research and practitioner-based initiatives have developed and tested e-metrics in public, academic, consortia, and other library settings since 1998 (see "The E-Metrics Playing Field," this page). These efforts created a wide range of e-metrics to measure the usage of library network services and resources. While there are differences across library types, general agreement emerged on the need for e-metrics in four core areas. E-metrics in the area of vendor data include information on commercial sessions and searches and items examined by patrons who access database, e-journal, ebook, and other licensed or purchased content. They include an e-metric that measures turnaways, which are rejected sessions caused by exceeding a database's simultaneous user limit. Network service data include new services or extensions of traditional library services into the networked environment. This can include virtual reference transactions conducted through email, a web site, or other network-based tools. It can also include counts of participants and hours of instruction in information technology. E-metrics in the area of network resources include the number of web-based resources such as digital collections and other web materials. E-metrics for expenditures count electronic materials costs relating to databases, e-journals, and ebooks.

Manager's edge

Besides giving funders an accurate picture of library usage, good data help in decision-making. Good data can help library managers determine whether they want to continue to develop and/or support services and resources. For example, a library manager could use e-metrics to examine numbers of Internet sessions and searches, full-content units examined, turnaways, and electronic materials expenditures. With these e-metrics, library managers can learn how much their electronic materials cost, how often and in what way they are used, what the resources cost on a per item basis (session, download, title), and whether the library has enough capacity to meet user demands for licensed resources (e.g., turnaways). Finally, managers will be able to determine the overall cost-effectiveness and efficiency of each electronic resource and service. Such data will be useful in vendor negotiations. Another example involves the "visitor" suite of e-metrics, such as virtual visits, percentage of virtual visits to total visits, and total visits. With these, library managers can determine the number of visitors to the library who are not physical visitors and over time track to what extent library use is changing. Such data will inform strategy about what form of use to promote (to bring more people into the buildings or a new portal) or what resources to reallocate (to offer more services/resources electronically or hire another librarian for the reference desk).

Going national

In a separate but important issue, librarians must get at least some of these e-metrics incorporated into existing state and national library data collection systems. The Association of Research Libraries and the Public Library Data Service now incorporate some e-metrics into their annual surveys. However, these surveys collect data from limited populations of libraries. There is a real need to incorporate these e-metrics into larger survey and data collection efforts. A good example is the annual public library surveys that state library agencies conduct with all public libraries in their states (of which some data are reported nationally through the Federal-State Cooperative System). Now that tested and defined e-metrics exist, library managers and staff need to determine which e-metrics can best help describe their library's network services and resources. They need to decide how best to use the data from the e-metrics to present library usage to stakeholders. And they must figure out which e-metrics are most appropriate for resource allocation and decision-making processes within the library. Likewise, states and those involved in national data collection efforts also need to consider these issues.

An e-metrics future

Librarians will become more comfortable measuring the use of network services and resources as they use e-metrics. The EMIS site is designed to aid in this transition by serving both instructional and informational roles. Work on it is ongoing, and it continues to undergo improvements. Visit www.ii.fsu.edu/emis to access the beta site. We welcome comments and suggestions regarding the site's design and content. In the short term, there is a need to work with vendors to develop standard reporting and analysis techniques. There is also a need to provide librarians with tools that help them access, analyze, and use vendor data across vendors. In the longer term, we need to develop measures that go beyond simply measuring usage to demonstrate the impact, benefits, and quality of library network services and resources. It's information that everyone needs to know.

---

John Carlo Bertot is Professor and Associate Director, Charles R. McClure is Francis Eppes Professor and Director, and Joe Ryan is Senior Research Associate, Information Use Management and Policy Institute, School of Information Studies, Florida State University, Tallahassee. Denise M. Davis is Program Manager, Library Development, Oregon State Library, Salem