Evaluating the performance of geographical locations in scientific networks with an aggregation - randomization - re-sampling approach (ARR)

Knowledge creation and dissemination in science and technology systems is perceived as a prerequisite for socio-economic development. The efficiency of creating new knowledge is considered to have a geographical component, i.e. some regions are more capable in scientific knowledge production than others. This article shows a method to use a network representation of scientific interaction to assess the relative efficiency of regions with diverse boundaries in channeling knowledge through a science system. In a first step, a weighted aggregate of the betweenness centrality is produced from empirical data (aggregation). The subsequent randomization of this empirical network produces the necessary Null-model for significance testing and normalization (randomization). This step is repeated to yield higher confidence about the results (re-sampling). The results are robust estimates for the relative regional efficiency to broker knowledge, which is discussed along with cross-sectional and longitudinal empirical examples. The network representation acts as a straight-forward metaphor of conceptual ideas from economic geography and neighboring disciplines. However, the procedure is not limited to centrality measures, nor is it limited to spatial aggregates. Therefore, it offers a wide range of application for scientometrics and beyond.

💡 Research Summary

The paper tackles the longstanding question of whether and how geographic location influences the ability of regions to create and disseminate scientific knowledge. Building on the premise that knowledge flows can be represented as a network of collaborations, citations, or co‑patents, the authors develop a three‑step analytical framework—Aggregation, Randomization, and Re‑sampling (ARR)—to quantify the relative “knowledge‑broker” efficiency of any spatial unit, be it a city, state, or country.

In the first step (Aggregation), an empirical network is constructed from bibliometric or patent data. Each node (e.g., author, institution, or paper) receives a betweenness centrality score, which measures how often the node lies on the shortest paths connecting all other node pairs. Betweenness is chosen because it directly captures a node’s potential to mediate information flow. The authors then aggregate the node‑level scores within each predefined geographic boundary. Aggregation is weighted: the centrality of a node is multiplied by a factor reflecting its scientific output (such as number of publications, citation count, or funding amount) before averaging. The result is a raw regional efficiency index that reflects both structural position in the network and productive capacity.

The second step (Randomization) creates a null distribution against which the raw index can be judged. The original network’s degree sequence is preserved while edges are repeatedly rewired using a degree‑preserving edge‑swap algorithm. This process generates a large ensemble of randomized networks that share the same local connectivity patterns but lack the specific higher‑order structure of the empirical network. For each randomized network the same aggregation procedure is applied, yielding a distribution of “expected” regional scores under the hypothesis of no systematic geographic advantage.

The third step (Re‑sampling) repeats the randomization many thousands of times, effectively bootstrapping the null distribution. From this distribution the authors compute a standardized z‑score for each region, a 95 % confidence interval, and a p‑value indicating the probability that the observed efficiency could arise by chance. By increasing the number of resamples, the statistical power and robustness of the results are enhanced, especially in sparse networks or when data are unevenly distributed across regions.

The methodology is illustrated with two empirical case studies. The first examines U.S. metropolitan areas using a co‑authorship network spanning 2000–2015. Results show that Silicon Valley, Boston, and Chicago consistently rank in the top quartile of broker efficiency, reflecting the concentration of research institutions, venture capital, and high‑tech firms. Temporal analysis reveals that policy interventions (e.g., the National Science Foundation’s Innovation Corps) correspond with measurable jumps in the broker scores of targeted regions. The second case focuses on European Union member states using a patent co‑ownership network. Germany and Sweden emerge as strong brokers, while many Eastern European countries display low scores. The authors link these patterns to the EU’s Framework Programme funding allocations, demonstrating that targeted research grants can shift a country’s position in the knowledge‑flow network.

In the discussion, the authors highlight several strengths of the ARR approach. First, betweenness centrality provides an intuitive, theory‑grounded metric for mediation potential. Second, degree‑preserving randomization controls for trivial degree effects, ensuring that observed regional advantages are not merely artifacts of having more connections. Third, extensive resampling yields reliable confidence intervals and facilitates hypothesis testing across multiple spatial scales. They also acknowledge limitations: betweenness focuses on shortest paths and may underestimate the role of longer, multi‑step diffusion processes; the randomization may disrupt higher‑order structures such as clustering or modularity, potentially biasing the null model; and the choice of weighting scheme can materially affect results, requiring careful sensitivity analysis.

The paper concludes that ARR offers a versatile, reproducible tool for scientometric evaluation of geographic performance. Because the framework is agnostic to the specific centrality measure, data source, or spatial aggregation, it can be adapted to sector‑specific networks (e.g., biotech collaborations), corporate R&D alliances, or even interdisciplinary knowledge maps. The authors provide open‑source code and a detailed workflow to encourage adoption by researchers, policy analysts, and funding agencies seeking evidence‑based insights into where and how scientific knowledge is most effectively brokered across space.