The differing meanings of indicators under different policy contexts. The case of internationalisation
📝 Abstract
In this chapter we build upon Moed’s conceptual contributions on the importance of the policy context when using and interpreting scientometric indicators. We focus on the use of indicators in research evaluation regarding internationalisation policies. The globalization of higher education presents important challenges to institutions worldwide, which are confronted with tensions derived from the need to respond both, to their local necessities and demands while participating in global networks. In this context, indicators have served as measures for monitoring and enforcing internationalisation policies, in many cases interpreting them regardless of the policy context in which they are enforced. We will analyse three examples of indicators related to internationalisation. The first one is about international collaborations, under the assumption that a greater number of internationally co-authored publications will benefit a national science system as it will result in higher citation impact. The second one relates to the promotion of English language as the dominant language of science. The third case analyses how different policy contexts shape the selection and construction of indicators, sometimes in a partial way which does not properly reflect the phenomenon under study. The examples illustrate that the interpretation and policy implications of the ‘same’ S&T indicators differ depending on specific contexts.
💡 Analysis
In this chapter we build upon Moed’s conceptual contributions on the importance of the policy context when using and interpreting scientometric indicators. We focus on the use of indicators in research evaluation regarding internationalisation policies. The globalization of higher education presents important challenges to institutions worldwide, which are confronted with tensions derived from the need to respond both, to their local necessities and demands while participating in global networks. In this context, indicators have served as measures for monitoring and enforcing internationalisation policies, in many cases interpreting them regardless of the policy context in which they are enforced. We will analyse three examples of indicators related to internationalisation. The first one is about international collaborations, under the assumption that a greater number of internationally co-authored publications will benefit a national science system as it will result in higher citation impact. The second one relates to the promotion of English language as the dominant language of science. The third case analyses how different policy contexts shape the selection and construction of indicators, sometimes in a partial way which does not properly reflect the phenomenon under study. The examples illustrate that the interpretation and policy implications of the ‘same’ S&T indicators differ depending on specific contexts.
📄 Content
The differing meanings of indicators under different policy contexts. The case of internationalisation
Nicolas Robinson-Garcia1 and Ismael Ràfols2,3
1 Delft Institute of Applied Mathematics (DIAM), TU Delft, Delft, Netherlands 2 INGENIO (CSIC-UPV), Universitat Politècnica de València, Valencia, Spain 3 Centre for Science and Technology Studies (CWTS), Leiden University, Leiden, Netherlands
Unrevised version to be published in the book “Evaluative informetrics – the art of metrics- based research assessment. Festschrift in honour of Henk F. Moed” edited by Cinzia Daraio and Wolfgang Glänzel
Abstract In this chapter we build upon Moed’s conceptual contributions on the importance of the policy context when using and interpreting scientometric indicators. We focus on the use of indicators in research evaluation regarding internationalisation policies. The globalization of higher education presents important challenges to institutions worldwide, which are confronted with tensions derived from the need to respond both, to their local necessities and demands while participating in global networks. In this context, indicators have served as measures for monitoring and enforcing internationalisation policies, in many cases interpreting them regardless of the policy context in which they are enforced. We will analyse three examples of indicators related to internationalisation. The first one is about international collaborations, under the assumption that a greater number of internationally co-authored publications will benefit a national science system as it will result in higher citation impact. The second one relates to the promotion of English language as the dominant language of science. The third case analyses how different policy contexts shape the selection and construction of indicators, sometimes in a partial way which does not properly reflect the phenomenon under study. The examples illustrate that the interpretation and policy implications of the ‘same’ S&T indicators differ depending on specific contexts.
Introduction The development and growth of the field of evaluative scientometrics cannot be understood without the fundamental contributions of Henk Moed. Along with his colleagues at the University of Leiden Centre for Science and Technology Studies (CWTS), he became a key player on establishing the basic pillars for the use of bibliometric indicators for research assessment (Moed, Bruin, & Leeuwen, 1995; Moed, Burger, Frankfort, & Van Raan, 1985). Moed’s work has been characterized by a critical notion on the use of indicators. He was one of the first to point out potential problems derived from the use of the Impact Factor for research assessment (Moed & van Leeuwen, 1996; Moed & Van Leeuwen, 1995), or the limitations of scientometrics when assessing the citation impact of non-English literature (Leeuwen, Moed, Tijssen, Visser, & Raan, 2001) among others. His two single-authored books ( Moed, 2005, 2017b), essential readings for anyone interested on the field, are characterized by an open-minded and pedagogical tone which reflects a critical and constructive view of evaluative scientometrics.
2
2
In his latest book, Moed proposes shifting away from a ‘narrow’ evaluative use of indicators to a more analytical one. He warns that “[t]o the extent that in a practical application an evaluative framework is absent or implicit, there is a vacuum, that may be easily filled either with ad-hoc arguments of evaluators and policy makers, or with un-reflected assumptions underlying informetric tools” (Moed, 2017, p. 29). In his view, the selection of indicators should be made within the ‘policy context’ in which they are going to be implemented (Moed & Halevi, 2015). Building upon this body of work, in this chapter we aim at further exploring this ‘analytical’ perspective on the use of scientometrics. We stress that context will not only provide the appropriate framework for the selection of indicators, but also for their interpretation, moving from a universal interpretation of indicators to a context- dependent one.
At this point, it is important to distinguish between policy context and adapting the indicators to a given context, what Waltman (2019a, 2019b) refers to as ‘contextualised scientometrics’. In the latter case, context is understood as a means to ensure transparency, and facilitate a better understanding on how the indicator is constructed and adapted to specific fields, countries or languages. The purpose in ‘contextualised scientometrics’ is to allow the user to grasp the limitations and biases inherent to scientometric indicators so that they are not misinterpreted due to technical and conceptual assumption on what the indicator is measuring. This is the line of thought followed by Gingras (2014) when defining the three desirable characteristics of a well- designed indicator: 1) adequacy for the object it measures, 2)
This content is AI-processed based on ArXiv data.