Grand challenges in altmetrics: heterogeneity, data quality and dependencies

📝 Abstract

As uptake among researchers is constantly increasing, social media are finding their way into scholarly communication and, under the umbrella term altmetrics, were introduced to research evaluation. Fueled by technological possibilities and an increasing demand to demonstrate impact beyond the scientific community, altmetrics received great attention as potential democratizers of the scientific reward system and indicators of societal impact. This paper focuses on current challenges of altmetrics. Heterogeneity, data quality and particular dependencies are identified as the three major issues and discussed in detail with a particular emphasis on past developments in bibliometrics. The heterogeneity of altmetrics mirrors the diversity of the types of underlying acts, most of which take place on social media platforms. This heterogeneity has made it difficult to establish a common definition or conceptual framework. Data quality issues become apparent in the lack of accuracy, consistency and replicability of various altmetrics, which is largely affected by the dynamic nature of social media events. It is further highlighted that altmetrics are shaped by technical possibilities and depend particularly on the availability of APIs and DOIs, are strongly dependent on data providers and aggregators, and potentially influenced by technical affordances of underlying platforms.

💡 Analysis

As uptake among researchers is constantly increasing, social media are finding their way into scholarly communication and, under the umbrella term altmetrics, were introduced to research evaluation. Fueled by technological possibilities and an increasing demand to demonstrate impact beyond the scientific community, altmetrics received great attention as potential democratizers of the scientific reward system and indicators of societal impact. This paper focuses on current challenges of altmetrics. Heterogeneity, data quality and particular dependencies are identified as the three major issues and discussed in detail with a particular emphasis on past developments in bibliometrics. The heterogeneity of altmetrics mirrors the diversity of the types of underlying acts, most of which take place on social media platforms. This heterogeneity has made it difficult to establish a common definition or conceptual framework. Data quality issues become apparent in the lack of accuracy, consistency and replicability of various altmetrics, which is largely affected by the dynamic nature of social media events. It is further highlighted that altmetrics are shaped by technical possibilities and depend particularly on the availability of APIs and DOIs, are strongly dependent on data providers and aggregators, and potentially influenced by technical affordances of underlying platforms.

📄 Content

Haustein, S. (2016). Grand challenges in altmetrics: heterogeneity, data quality and dependencies. Scientometrics. doi: 10.1007/s11192-016-1910-9 1 Grand challenges in altmetrics:
heterogeneity, data quality and dependencies Stefanie Haustein Keywords: Big data; Data integration; Research and innovation policy; Data quality; Comparability; Standardization; Concordance tables; Modularization; Interoperability; Research assessment Abstract As uptake among researchers is constantly increasing, social media are finding their way into scholarly communication and, under the umbrella term altmetrics, were introduced to research evaluation. Fueled by technological possibilities and an increasing demand to demonstrate impact beyond the scientific community, altmetrics received great attention as potential democratizers of the scientific reward system and indicators of societal impact. This paper focuses on current challenges of altmetrics. Heterogeneity, data quality and particular dependencies are identified as the three major issues and discussed in detail with a particular emphasis on past developments in bibliometrics. The heterogeneity of altmetrics mirrors the diversity of the types of underlying acts, most of which take place on social media platforms. This heterogeneity has made it difficult to establish a common definition or conceptual framework. Data quality issues become apparent in the lack of accuracy, consistency and replicability of various altmetrics, which is largely affected by the dynamic nature of social media events. It is further highlighted that altmetrics are shaped by technical possibilities and depend particularly on the availability of APIs and DOIs, are strongly dependent on data providers and aggregators, and potentially influenced by technical affordances of underlying platforms. 1 Introduction Social media have profoundly changed how people communicate. They are now finding their way into scholarly communication, as researchers increasingly use them to raise their visibility, connect with others and diffuse their work (Rowlands, Nicholas, Russell, Canty, & Watkinson, 2011; Van Noorden, 2014). Scholarly communication itself has remained relatively stable; in the course of its 350-year history the scientific journal has not altered much. Even in the digital age, which has facilitated collaboration and increased the speed of publishing, the electronic journal article remains in essence identical to its print counterpart. Today, the peer-reviewed scientific journal still is the most important channel to diffuse scientific knowledge. In the context of the diversification of the scholarly communication process brought about by the digital era, social media is believed to increase transparency: ideas and results can be openly discussed and scrutinized in blog posts, some journals and designated platforms are making the peer-review process visible, data and software code are increasingly published online and reused, and manuscripts and presentations are being shared on social media. This diversification of the scholarly communication process presents both an opportunity and a challenge to the scholarly community. On the one hand, researchers are able to distribute various types of scholarly work and reach larger audiences; on the other hand, this leads to a further increase of information overload. At first, altmetrics were seen as an improved filter to overcome the information overload stemming from the diversification and increase in scholarly outputs (Priem, Taraborelli, Groth, & Neylon, 2010). In that sense, quite a few parallels exist between the development of bibliometrics and altmetrics: It is too much to expect a research worker to spend an inordinate amount of time searching for the bibliographic descendants of antecedent papers. It would not be excessive to demand that the thorough scholar check all papers that Haustein, S. (2016). Grand challenges in altmetrics: heterogeneity, data quality and dependencies. Scientometrics. doi: 10.1007/s11192-016-1910-9 2 have cited or criticized such papers, if they could be located quickly. The citation index makes this check practicable. (Garfield, 1955, p. 108) No one can read everything. We rely on filters to make sense of the scholarly literature, but the narrow, traditional filters are being swamped. However, the growth of new, online scholarly tools allows us to make new filters; these altmetrics reflect the broad, rapid impact of scholarship in this burgeoning ecosystem. (Priem et al., 2010, para. 1) While altmetrics rely on users of various social media platforms to identify the most relevant publications, datasets and findings, Garfield (1955) and before him Gross and Gross (1927) believed that citing authors would outperform professional indexers in identifying the most relevant journals, papers and ideas. Both altmetrics and citation indexing thus rely on collective intelligence, or

This content is AI-processed based on ArXiv data.