Research assessment, or the process that universities use to measure the value of published papers and other research outputs, plays a critical role in hiring, promotion, and tenure decisions. It can have a profound effect on the job stability and reputation of researchers. With such high stakes involved, it’s important for universities to get this process right—and librarians can help their institutions achieve this goal.

For years, the key metric that many institutions used in assessing the value of research was the Journal Impact Factor, a figure designed to measure the relative importance of a journal within its field.

Based on the average number of times the articles appearing within a journal have been cited in the last two years, the Journal Impact Factor was originally intended to help librarians figure out which academic journals were worth subscribing to. Yet, it soon became a simple proxy for ascribing value to a researcher’s work: If a paper was published in a journal with a high impact factor, then it was valued more highly.

However, this approach to research assessment is problematic. 

In using the Journal Impact Factor, “you’re putting too much emphasis on the venue where the article is published,” says Joseph Lerro, open research business development manager for Taylor & Francis Group. “You’re not considering the actual article itself.”

A better way

In 2012, a group of editors and publishers of scholarly journals met during the American Society for Cell Biology annual meeting in San Francisco to address the need for improving research assessment practices. The group developed a set of recommendations, referred to as the San Francisco Declaration on Research Assessment (DORA).

DORA called on universities and funding agencies to assess research on its own merits, rather than on the basis of the journal in which it’s published. 

In the wake of DORA, a number of alternative metrics—or “altmetrics”—have emerged to help institutions determine the value of a research article. These article-level metrics include the number of times an article has been viewed, cited, mentioned in the news or in policy documents, shared on social media, and even used in the course syllabi of professors.

“Measures like these are more indicative of what a research article can add to the scientific record,” Lerro observes.

How librarians can help

As information science experts who have a broad understanding of the research publishing landscape, librarians are well positioned to help their institutions use altmetrics to improve the research assessment process.

“One way they can assist is by providing support and education to the people who are conducting research reviews to help them understand the various metrics available and the pros and cons of using each one,” Lerro says. 

By familiarizing themselves with altmetric tools, creating databases or knowledge articles with research assessment resources, and having conversations with Research Office staff and assessment boards, librarians can ensure their institution is taking a more holistic approach to research assessment.

“Librarians can help their universities put together the most comprehensive system possible, one that paints a clearer picture of the practical, real-world impact of research outputs,” Lerro says.

Publishers also play a role

Academic publishers also play a key role in supporting better research assessment. For instance, Taylor & Francis and its F1000 open research platform display altmetrics for each published article to help researchers and institutions understand the impact of their work.

Lerro will be co-presenting an October 25 Library Journal webinar on the future of research assessment—and the role of librarians in shaping it—along with Liz Allen, director of strategic initiatives for F1000; Sheila Craft-Morgan, research impact librarian and assistant professor at Ohio State University; and Michael Levine-Clark, dean of the University of Denver Libraries.

During this free, one-hour event, titled “Responsible research assessment: How librarians can help drive change,” the panel will reflect on how research assessment works today—and how librarians can help transform it for the better.

To learn more or register, click here.

SPONSORED BY