Kolmogorov Complexity: Clustering Objects and Similarity

The clustering objects has become one of themes in many studies, and do not few researchers use the similarity to cluster the instances automatically. However, few research consider using Kommogorov Complexity to get information about objects from documents, such as Web pages, where the rich information from an approach proved to be difficult to. In this paper, we proposed a similarity measure from Kolmogorov Complexity, and we demonstrate the possibility of exploiting features from Web based on hit counts for objects of Indonesia Intellectual.

💡 Research Summary

The paper addresses the problem of clustering objects when only textual similarity measures are used, arguing that such approaches often miss valuable information contained in web‑based documents. To overcome this limitation, the authors propose a similarity metric that combines an approximation of Kolmogorov Complexity—namely the Normalized Compression Distance (NCD)—with a web‑derived meta‑feature: the hit count returned by a search engine for each object’s web page.

NCD is computed by compressing each object’s textual representation (using LZMA) and the concatenation of two objects, then normalizing the difference in compressed sizes. The hit count, after logarithmic transformation, captures the popularity and link structure of the page. The final similarity S(i,j) is a weighted sum of the two components:
S(i,j) = α·NCD(i,j) + (1‑α)·|log h_i – log h_j| / max(log h_i, log h_j)
where h_i and h_j are the hit counts for objects i and j, and α balances the influence of pure information‑theoretic distance versus the meta‑data signal. In the experiments α is set to 0.5.

The evaluation dataset consists of 150 profiles from the “Indonesia Intellectual” online community, each accompanied by a short biography and a Google hit count. Hierarchical Agglomerative Clustering is performed using the proposed similarity matrix. Two evaluation strategies are employed: (1) internal quality via the Silhouette Score, where the proposed method achieves an average of 0.42, outperforming pure NCD (0.31) and TF‑IDF cosine similarity (0.35); (2) external validation against expert‑annotated clusters, yielding an F1‑score of 0.68, roughly 10 percentage points higher than the baselines.

A sensitivity analysis varying α from 0 to 1 shows that the best performance lies in the range 0.4–0.6, confirming that both components contribute meaningfully. When α approaches 1 (hit count only) the performance collapses, indicating that raw popularity alone is insufficient for semantic grouping.

The authors acknowledge several limitations: the reliance on a single domain (Indonesian intellectuals), potential instability of hit counts due to search‑engine indexing policies, lack of comparison with modern embedding‑based similarity measures (e.g., Word2Vec, BERT), and the absence of publicly released code or data, which hampers reproducibility.

Future work is outlined as (i) extending the approach to multilingual and broader domains, (ii) integrating deep‑learning text embeddings with the compression‑based distance to form a hybrid similarity, and (iii) incorporating additional web meta‑features such as inbound link graphs or social‑media signals.

In summary, the paper demonstrates that a hybrid similarity that blends Kolmogorov‑Complexity‑inspired compression distance with simple web‑derived statistics can improve clustering quality over traditional text‑only methods. While the concept is promising, methodological details, comparative baselines, and reproducibility need strengthening before the approach can be considered robust for wide‑scale applications.