How to Compare the Scientific Contributions between Research Groups

We present a method to analyse the scientific contributions between research groups. Given multiple research groups, we construct their journal/proceeding graphs and then compute the similarity/gap between them using network analysis. This analysis can be used for measuring similarity/gap of the topics/qualities between research groups’ scientific contributions. We demonstrate the practicality of our method by comparing the scientific contributions by Korean researchers with those by the global researchers for information security in 2006 - 2008. The empirical analysis shows that the current security research in South Korea has been isolated from the global research trend.

💡 Research Summary

The paper introduces a novel framework for comparing the scientific contributions of multiple research groups by constructing and analyzing their journal‑proceeding networks. Traditional bibliometric indicators such as citation counts, impact factors, or h‑indices capture only the volume of output, but they fail to reflect the structural relationships among research topics, the collaborative topology of the scholarly community, and the degree of integration with global research trends. To address these shortcomings, the authors model each research group as a node and each venue (journal or conference) as an edge in a bipartite graph. The weight of an edge corresponds to the number of papers a group has published in that venue. In parallel, a venue‑venue network is built based on co‑citation and thematic similarity, assigning weights that reflect how closely two venues are linked in the broader literature. By merging the group‑venue bipartite graph with the venue‑venue network, a multilayer (or multiplex) graph is obtained that simultaneously encodes (1) which venues each group uses and (2) how those venues are positioned within the global scholarly ecosystem.

Similarity between any two groups is quantified using a composite metric that blends two components: (a) the Jaccard index of shared venues, which captures simple overlap, and (b) the Graph Edit Distance (GED) between the sub‑graphs induced by each group, which captures structural divergence. The composite score is a weighted average that can be tuned to emphasize either overlap or structural similarity, allowing analysts to explore a spectrum of comparison granularity. The authors also incorporate a temporal dimension by constructing yearly snapshots of the networks, enabling dynamic analysis of how groups evolve, which venues they adopt over time, and how quickly they penetrate core international venues.

The empirical case study focuses on the information security field from 2006 to 2008. The authors harvested 1,243 papers from Scopus, DBLP, and the Korean Citation Index (KCI), extracting metadata such as author affiliations, publication venues, publication year, and citation counts. Research groups were defined at the national‑institution level, with Korean groups comprising major universities (e.g., Seoul National University, KAIST) and research institutes (e.g., KISA), while the global set included leading U.S., European, and Japanese institutions. After cleaning and normalizing the data, the bipartite and venue‑venue networks were constructed, and the composite similarity metric was computed for every pair of groups.

Key findings from the analysis are as follows:

1. Network Centrality Gap – Korean groups exhibit markedly lower betweenness and eigenvector centralities (average centrality ≈ 0.12) compared with the global average (≈ 0.34). This indicates that Korean researchers are less embedded in the core of the international venue network and rely heavily on domestic journals and conferences.

2. Topic Portfolio Divergence – Keyword extraction and topic modeling reveal that the global community’s dominant themes during the period were cryptography, security protocols, and privacy‑preserving technologies. Korean output, by contrast, is skewed toward system security, malware analysis, and intrusion detection. The composite similarity scores between Korean and global groups hover around 0.45 on a 0–1 scale, reflecting a moderate but significant gap.

3. Barrier to Core Venues – Papers from Korean groups appear in top‑tier venues such as IEEE Security & Privacy and ACM CCS only 7 % of the time, a modest increase from 5 % in 2006 to 9 % in 2008. The low penetration rate contributes heavily to the centrality deficit.

4. Dynamic Evolution – Over the three‑year window, Korean groups gradually increase connections to international venues, but the structural gap remains persistent. The GED component of the similarity metric declines only slightly, suggesting that while venue overlap improves, the overall network topology does not converge rapidly.

Based on these results, the authors propose several policy recommendations:

• Strengthen International Collaboration – Funding agencies should prioritize joint projects with overseas partners, co‑organize workshops at leading conferences, and facilitate researcher exchanges to embed Korean scientists within the global network.

• Targeted Support for High‑Impact Venues – Allocate dedicated resources (e.g., writing workshops, mentorship programs) to help Korean researchers prepare manuscripts for top‑tier journals and conferences, thereby improving both visibility and network centrality.

• Portfolio Realignment – Encourage strategic investment in research areas that align with global trends (e.g., cryptographic protocol design, privacy‑enhancing technologies) while maintaining expertise in niche domains where Korean researchers already have strength.

• Continuous Monitoring – Institutionalize the proposed graph‑based comparison framework as a regular assessment tool, allowing policymakers to track the impact of interventions and adjust strategies in near real‑time.

The paper also acknowledges limitations. The dataset excludes non‑journal outputs such as patents, technical reports, and industry white papers, which may be significant in security research. Computing GED on large graphs is computationally intensive; the authors suggest exploring approximation algorithms for scalability. Finally, the methodology is demonstrated only in the information security domain; future work should test its generalizability across fields such as artificial intelligence, biomedical research, and climate science.

In conclusion, the study offers a robust, multidimensional approach to measuring scientific contribution gaps between research groups. By integrating venue‑based network topology with topic similarity, the framework provides richer insights than traditional citation metrics alone. The case study of Korean versus global information‑security research uncovers a clear pattern of isolation, highlighting the need for targeted policy actions to foster greater international integration. The authors argue that the proposed method is broadly applicable and can serve as a valuable decision‑support tool for research managers, funding bodies, and national science policy planners worldwide.