Introduction

Until recently, measurement of scholarly influence was the exclusive domain of specialized citation indexing tools, relying principally on citations to articles published in select journals to construct an understanding of individual scholar reputation. Parallel developments in recent years have brought analytics and Internet search optimization tools to any savvy Internet user. These developments have given rise to altmetrics, the process of expanding the measurement of scholarly impact to include the social web, beyond traditional citations. The convergence of the tools and models of the past with the analytical tools of the online environment opens a space for innovation, and poses an interesting challenge for libraries to define a role.

For librarians to shape a service that may assist scholar-practitioners and graduate students to find their way with altmetrics and scholarly promotion, more needs to be known about how these groups perceive and engage with the tools available to them. Our research explores this terrain, querying participants about what issues they face when trying to establish, grow, or measure a scholarly presence on the web, as well as how they negotiate these issues. From this we discern and suggest ways in which academic librarians can assist scholar-practitioners and students to create, discover, and manage elements of online reputation using traditional and emerging tools for measuring influence.

Literature Review

The idea of altmetrics dates from 2010, when Jason Priem, a doctoral candidate at the University of North Carolina at Chapel Hill, first used the term on Twitter. An influential manifesto followed, articulating the limitations of traditional filters of academic quality: article citations and journal impact factor (Priem, Taraborelli, Groth, & Neylon, 2010). Subsequently, Heather Piwowar, researcher and altmetrics advocate, cited four potential advantages to altmetrics:

A more nuanced understanding of impact, showing us which scholarly products are read, discussed, saved and recommended, as well as cited.
Often more timely data, showing evidence of impact in days instead of years.
A window on the impact of web-native scholarly products like datasets, software, blog posts, videos, and more.
Indications of impacts on diverse audiences, including scholars, but also practitioners, clinicians, educators, and the general public (Piwowar, 2013)

Despite the advantages articulated by Piwowar, some researchers have balked at altmetrics, questioning if tweets and blog post mentions are a real indicator of impact (Scott, 2012). Others are concerned that altmetric data can be manipulated. One study tested how easy it is to game Google Scholar metrics, concluding it is “simple, easy, and tempting” (Howard, 2013). However, attempting to game one’s scholarly influence is nothing new in academia; as the “publish or perish” model continues to weigh on researchers, there has been an increasing number of paper retractions in journals due to research fraud and increased journal vigilance (Fang, Steen, & Casadevall, 2012; Steen, 2011).

Recognizing both the value and the uncertainty of altmetrics, many scholars and librarians - including the authors of this paper - choose to see traditional informetrics indicators and altmetrics as complementary (Costas, Zahedi, & Wounters, 2015). Rousseau and Yee (2013) suggest that Cronin & Weaver’s 1995 term “influmetrics” is a more useful term than altmetrics, while “allmetrics” is used by Plum Analytics, a for-profit scholarly analytics company recently purchased by EBSCO.

In our review of the literature, we noted that institutions focussed on applied research were not represented in altmetrics discussions. As librarians from universities with emphasis on applied programming, this lack of representation surprised us; this type of institution is home to many scholar-practitioners who undertake significant work that is often not published through traditional channels. For example, non-governmental organization reports, briefing notes, papers of all kinds, and instructional resources are frequently published by applied scholar-practitioners, but up until recently have been difficult to track for impact.

Scholar-practitioners have been defined as a group who characteristically see their work “in relation to broader organizational, community, political, and cultural contexts [and] explicitly reflect on and assess the impact of their work” (McClintock, 2004). In our applied and teaching-focused institutional settings, we consider that scholar-practitioners include both those who maintain professional practice in their teaching area, and those for whom teaching itself is the practice.

Aims

As altmetrics now offers a way to gauge the level of influence that diverse types of published scholarly work may have, we believe there is an opportunity for libraries to define new services related to scholarly profile curation and management. In order to derive greatest benefit from altmetric tools, the intentional development of an online scholarly profile is necessary. “Online scholarly profile” refers to a curated representation of the digital footprints left by scholar-practitioners on the web. This might include published articles, books, Tweets, blogs, datasets, reports, comments, presentations, Academia.edu profiles, or any other data that is published online by a scholar-practitioner. Librarians have the opportunity to advise scholar-practitioners in how to most effectively keep track of and represent the influence of all these types of online evidence, and the implications of tool choice.

In order to shape a new evidence-based scholarly profile service at our libraries, we sought to assess the awareness and needs of our communities, asking the following questions:

What issues do scholars and graduate students who are also practitioners face when trying to establish, grow, or measure a scholarly presence on the web?
How do scholars and graduate students who are also practitioners negotiate issues related to establishing or growing a scholarly presence on the web?
In what ways can academic librarians assist scholar-practitioners to create and manage online reputation using traditional and emerging tools for measuring influence?

Methods

In order to explore scholar-practitioner and graduate student attitudes and practices related to altmetrics and online scholarly profiles, semi-structured interviews occurred with 18 scholar-practitioners and 5 graduate students from two institutions: Vancouver Island University and Royal Roads University. Interviews ranged from 45 minutes to 1.5 hours.

Participants were chosen based on factors that included publication record, including both traditional and alternative channels of dissemination, and demonstrated interest in new modes of scholarly communication and networking. At Vancouver Island University, the annual institutional report on scholarly activity, which lists the scholarly output and service of all faculty members, was consulted. From this guide we identified individuals who were actively engaged in scholarship. We then researched these scholars through publicly accessible tools like Google Scholar, Mendeley, and Impact Story to see if the work of these individuals was represented. Subsequently, we contacted these potential participants, requested an interview, and offered to show individuals the impact of their work using reports and examples drawn from altmetrics tools, as available. Graduate student participants were recruited via snowball sampling.

Scholar-practitioners and students who chose to participate were queried about their impressions of traditional scholarly metrics, engagement with altmetrics and social media, and opinions regarding academic library potential roles in providing services related to altmetrics and scholarly reputation. Participants were also shown the impact of their scholarly work using reports drawn from altmetrics tools, as available. Traditional to emerging metrics tools including Web of Science, Google Scholar, Mendeley, Impact Story, and Plum Analytics were demonstrated and discussed. Interview data was transcribed using MS Word, printed, and then coded by hand. The authors then met to discuss the themes that emerged from the transcripts.

Results

Participants were first queried about how they use the Internet professionally, and their knowledge of impact and altmetrics. Every participant had searched for his or her own name through Google for professional purposes. Very few of the participants were strategic about their online presence, not having built Google Scholar profiles, searched their names through altmetrics tools, or otherwise had a systematic approach to online identity. There was general awareness of impact related to journal impact factor, and a general perception that this calculation was important, although most participants could not articulate why.

The term ‘altmetrics,’ and associated tools, were new to most participants. Participants affirmed the perception that altmetrics can be of value in helping to craft a story of the value of their research and its diverse outputs, as opposed to simplistic benchmarking. When participants had prior knowledge of altmetrics tools, it tended to be very narrow and deep, and perhaps field-specific. For example, one interviewee told us about a tool called the Carbon Capture Report (http://wwwcarboncapturereport.org). This site tracks and ranks the tone and activities of individuals who post and publish about climate change and alternative energy in social media.

Whether participants felt that altmetrics and scholarly profile tools were important to their own careers depended on several variables:

Career stage: Younger scholar-practitioners recognized that these tools are becoming increasingly important. Several participants close to retirement also recognized value, not necessarily for themselves, but for their graduate students. We had several graduate students participate in our study at the urging of their supervisors.
Career aspirations (or lack thereof): Several participants did not plan on changing jobs, so felt that altmetrics were unimportant to them.
Institutional structure and value placed on research: Most Vancouver Island University participants commented that the lack of a tenure-based advancement system at their institution, and weak incentives to pursue an active program of scholarship, corresponded to a low incentive to track influence.
Home discipline: Scientists in our sample were more familiar with the tools that we presented and quickly understood their importance. We believe this to be related to a predisposition in the sciences toward quantitative methods. For example, upon hearing about ORCid (http://orcid.org) numbers (persistent digital identifier numbers for individual researchers), one chemist in our study immediately perceived a parallel with Chemistry Abstract Service (CAS) numbers.
Major grant applicants: Those participants who actively applied for major research grants were enthusiastic about the possibility of demonstrating their impact beyond traditional means.

Thus, to what extent interviewees were engaged with scholar-practitioner profile and altmetrics tools depended not only on whether people knew about them or not, but also on the perception of direct relevance. Philosophical beliefs about social media and privacy boundaries also factored into decisions to use profile and altmetric tools. Some participants were averse to posting about themselves online, while others felt it impossible to separate their private and public selves. Other participants mentioned that how one is expected to behave within particular disciplines may play a role. In more traditional disciplines, promoting one’s own work on social media would be frowned upon, and the person labelled a braggart. In other fields, self-promotion is completely accepted.

By far the most common barrier to the use of these tools that participants noted was time. It takes substantial time to set up, track, and ensure that profiles are continuously updated. Some participants mentioned that they delegated this type of work to graduate students. Other participants mentioned not having the time to learn about and determine which particular tool(s) is best for their disciplines. The stage of development of tools also was influential in how and whether people decided to engage with tools. With high production values, including effective and informative visual displays customized to the scholar, PlumX (https://plu.mx) was a favourite tool among participants. Sites that looked less polished and had fewer features inspired less confidence among participants.

Participants expressed some skepticism over what altmetrics tools could accomplish, how these sorts of measures might be ‘gamed,’ and how to make meaningful comparisons across or within disciplines. Also, through a labour-management lens, there was some concern about faculty being reduced to numbers and then compared to one another.

Many of the participants were surprised to see their professional identities represented online, having never or seldom uploaded professional content to the web themselves. In many of these cases, conference organizers, journal staff, or co-authors uploaded abstracts, conference presentations, and articles without active participation from our study participants. Seeing how evidence of their work is already represented online, whether they like it or not, led participants to feel a need for guidance and assistance. When the interviewees were asked if such a service might be situated in the library, all affirmed a role for librarians. There was also general support for the idea that indicators of influence related to scholarly publication could be more diverse than citations, capturing different kinds of use, and that the influence of non-journal publications could also be indicated.

Discussion

A number of ways in which librarians can support scholar-practitioners and graduate students with profile and altmetrics tools have been identified through this research:

Personalized Support

As outlined above, participants’ attitudes toward altmetrics and a potential library service in this area varied substantially based on stage of career, field of scholarly activity, institutional value placed on research and publishing in faculty role, formality of institutional promotion requirements, dependence on traditional impact measures to obtain and retain funding, time, awareness of tools, perceived readiness of tools, and disciplinary view of tools. Consequently, users’ objectives will vary greatly, and any service needs to be highly personalized in nature. While an online library guide to altmetrics and curating scholarly profiles online would be a useful tool to introduce users to the concept of altmetrics and scholarly profiles, and may provide a starting point for the enterprising and self-motivated, it is only a first step.

Establish Goals

Participants in this study listed several different reasons for wanting to build and measure a scholarly profile online: tracking the influence of their work outside of traditional journal literature, building an audience outside of the academy, and making connections to other researchers with interest in the same area but perhaps in other disciplines, and attracting graduate students and funders. Librarians can highlight the opportunities and strengths afforded by various tools, and recommend particular services based on the goals of individuals.

Tools

As professionals often connected with both social media and scholarly communication, academic librarians are in a prime spot to act as guides in orienting scholar-practitioners and students to the altmetrics environment and social media conventions for academic purposes.

One of the main places where librarians can assist scholar-practitioners and students who are starting out with establishing a scholarly presence online is to go over the tools available and make recommendations as to which to use. With so many tools and sites, participants asked: Which tools offer the features that I am interested in? Which ones are my colleagues using? What’s best for my discipline? One participant stated that developing a “road map” of exactly what tools to use, and what had to be done to maintain them effectively, would be helpful.

Orientation to the Altmetrics and Scholarly Promotion Landscape

Librarians can provide education and guidance on specific actions to take and tools to use, but we also have a role in encouraging users in developing nuanced perceptions of social media. Research participants expressed a wide spectrum of attitudes toward social media and alternative channels of disseminating their work. These attitudes ranged from seeing social media as fraught with and characterized by flame wars and egocentrism to perceiving it as a critical, emerging venue for dissemination.

A key hurdle to utilizing social media is an aversion to self-promotion. A majority of participants in this study commented on being uncomfortable with promoting their own work in social media, and creating what one participant referred to as a scholarly “egosystem.” Another participant worried about being “dismissed as a braggart.” Although this study examined only a small sample of 23 individuals, trends emerged related to self-promotion conventions within disciplines. Participants from interdisciplinary backgrounds, for example, had less concern about self-promotion than those in natural sciences. As all participants were of Canadian nationality, we couldn’t help but wonder if our stereotypical national politeness and aversion to limelight played a role. A cross-cultural study of how academics view online promotion would be useful in determining what demographics most influence participation. Regardless of the motivation for shying away from social media, librarians can assist service users by drawing a distinction between useful promotion of one’s own scholarly work, and egoism.

Preparing Users for the Tough Times

At the same time that they aspired to a broader audience, a handful of participants expressed fear that once released into the digital wilds beyond the academy, their work might be misinterpreted or misused. One participant explained that this already had happened to him: an article on climate change was cited as proof of “intelligent” design. Others expressed feelings of frustration and uncertainty about how online data they create might be misused. Issues with accounts being hacked, services spamming email accounts, and worries that research on controversial topics could be used against a scholar or their broader network (i.e., their graduate students) were also highlighted as serious concerns by participants. If librarians are going to assist our users in setting up profiles online and using various tools, we also share in a responsibility to prepare them for and assist when tough times arrive – including hacks, misrepresentation, and spamming. Making users aware of the potential downside of engaging with profile tools should be a part of any service the library provides.

Copyright

As scholarly work becomes more accessible and visible, scholar-practitioners and students have amplified responsibility to ensure that their published work is in compliance with copyright law. Librarians can assist by educating about Creative Commons resources and models, enhancing copyright awareness on campus, and reviewing work that individuals wish to release to the public.

Packaging Content

Many academics are accustomed to writing in a particular style for a scholarly audience. Work produced for this specialized audience will not necessarily be accessible to the general public. As one of our participants commented,

I can’t just take out a [research finding], plop it on the Internet, because you’re not going to reach a bigger audience. You’re probably going to reach people who already know about your work through citations and things like that. Translational things take time, repackaging the content for a wider audience. ... I think as a scientist you have an obligation to share knowledge in as many different formats as you possibly can.

Librarians can help scholar-practitioners and graduate students to identify their audience segments, and to present work in ways that are the most suitable for various forums.

Open Access (OA) Education

Conversations about scholarly profile flow naturally into a discussion of making work available in OA form, ensuring that the opportunity exists for scholarship to be viewed as widely as possible. Librarians may assist scholar-practitioners to assert their author rights with publishers in order to republish content in OA form, as well as provide education on suitable OA repositories. There is some confusion among scholar-practitioners on the subject of appropriate Open Access repositories in which to deposit content; for-profit sharing sites like Academia.edu or Research Gate are not in fact open, and do not comply with the Tri-Agency Open Access Policy on Publications (Government of Canada, 2015).

Promoting the New Service

In interviews, participants suggested a number of ways that librarians could reach researchers who would be interested in a library-led profile and impact service. Several suggested talking to people on campus concerned with knowledge mobilization, as these are likely to be natural allies. Knowledge mobilization is all about getting work out of the academy and making it useful in the community, so measuring the ways in which this happens is a relatively “easy sell” to these potential allies.

As mentioned above, many participants did not know that their work was already represented online. If scholar-practitioners are unaware of how their work is being disseminated and discussed in non-traditional venues, emailing publicly accessible impact reports can entice them to learn more and to take an active role in shaping their online profiles.

Aside from direct contact with people whose work is already represented online, librarians can approach scholar-practitioners who are working on research projects – particularly those who are in the planning stages. In addition to providing data management planning guidance, librarians can assist scholar-practitioners in planning where, when, and how to mobilize and track use of the knowledge being produced and disseminated.

Altmetrics are an excellent springboard to use to talk to graduate students and senior undergraduates, who are engaged in capstone projects, about beginning to curate a scholarly profile. While many of these students have not published in journals, they will often begin by giving posters or presentations at conferences. By uploading their posters to their institutional repository or their slides to a site like Slideshare, these works are made accessible online and can be tracked for views and downloads. Teaching graduate students to seed their profiles early and to deliberately apply online tools for understanding their scholarly influence serves them as they seek to establish themselves in their fields.

Conclusion

The opportunity for libraries going forward is to augment their existing role with respect to scholarly profiles: libraries have long provided access to and interpreted citation indexes, cited references, and the traditional scholarly publication cycle. The emerging scene challenges libraries to support scholar-practitioners and students to engage with a broader, volatile, and evolving environment in which much may be gained or forfeited depending on how reputation is curated.

Having demonstrated that an interest and need exists on our campuses for a library-led service related to scholarly profile and impact measurement, the next step in our research is to pilot a highly individualized consultation service for scholar-practitioners and students. As traditional library services change or disappear, we see this service as a natural extension of library expertise, and a meaningful way to provide outreach and support to our scholar-practitioners and graduate students.

References

Costas, R., Zahedi, Z., & Wounters, P. (2015). Do “altmetrics” correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective. Journal of the Association for Information Science and Technology, 66(10), 2003–2019.

Cronin, B., & Weaver, S. (1995). The praxis of acknowledgement: From bibliometrics to influmetrics. Revista Española de Documentación Científica, 18(2), 172–177. http://dx.doi.org/10.3989/redc.1995.v18.i2.654

Fang, F. C., Steen, R. G., & Casadevall, A. (2012). Misconduct accounts for the majority of retracted scientific publications. Proceedings of the National Academy of Sciences of the United States of America, 109(42), 17028–33. http://dx.doi.org/10.1073/pnas.1212247109

Government of Canada. Tri-Agencies. (2015). Tri-Agency Open Access Policy on Publications. Retrieved 14 March, 2016, from http://www.science.gc.ca/default.asp?lang=En&n=F6765465-1.

Howard, J. (2013). Rise of “altmetrics” revives questions about how to measure impact of research. Chronicle of Higher Education. Retrieved from https://chronicle.com/article/Rise-of-Altmetrics-Revives/139557/

McClintock, C. (2004). Scholar practitioner model. In Encyclopedia of Distributed Learning. Thousand Oaks, CA: SAGE. Retrieved April 20, 2016 from http://knowledge.sagepub.com/view/distributedlearning/n134.xml

Piwowar, H. (2013). Altmetrics: What, why, and where?”. ASIS&T Bulletin, 39(4), 8–9. Retrieved from https://www.asis.org/Bulletin/Apr-13/AprMay13_Piwowar.html

Priem, J., Taraborelli, D., Groth, P., & Neylon, C. (2010). Altmetrics: A manifesto. Retrieved February 15, 2016, from http://altmetrics.org/manifesto/

Rousseau, R., & Ye, F. Y. (2013). A multi-metric approach for research evaluation. Chinese Science Bulletin, 58(26), 3288-3290. http://dx.doi.org/10.1007/s11434-013-5939-3

Scott, N. (2012). Altmetrics are the central way of measuring communication in the digital age but what do they miss? In London School of Economics and Political Science Blog. Retrieved February 15, 2016, from http://blogs.lse.ac.uk/impactofsocialsciences/2012/12/17/scott-altmetrics-central-digital-whats-missing/

Steen, R. G. (2011). Retractions in the scientific literature: Is the incidence of research fraud increasing? Journal of Medical Ethics, 37(4), 249–253. http://dx.doi.org/10.1136/jme.2010.040923

Appendix – Interview Questions

Tell me your impressions of scholarly metrics as they exist now? How do you see yourself in this system?

Have you searched for yourself and your work online in the last year? How did you do so?

Have you tried to build awareness of your work online? If so, how? If not, why not?

Have you experimented in non-traditional venues? If not, why not?

Do you see scholarly metrics as being important to building your academic career? If so, how?

Do you have a professional presence in social media? What do you consider to be the advantages and disadvantages in terms of building reputation as a scholar-practitioner?

Do you think your discipline or field is receptive to innovation with respect to forms of publication and associated metrics?

How do you integrate (or separate) your scholarly and personal lives online? Describe this process (difficulties, technologies, etc.).

Midway through the interview, demonstrate altmetrics in Mendeley, Google Scholar, Reader Meter, Impact Story.org.

Give us your impression of these tools.

Would you be interested in meeting with other scholars/colleagues to discuss possible uses of new ways to measure influence across disciplines and emerging forms of scholarly communication? 

Would you be interested in a workshop focused on exploring how to use library or social tools to assess your influence as a scholar-practitioner?

Would you be interested in an appointment-based service where you could consult with a librarian on a practical strategy for managing and monitoring your online presence as a scholar-practitioner?