kLog: A Language for Logical and Relational Learning with Kernels

We introduce kLog, a novel approach to statistical relational learning. Unlike standard approaches, kLog does not represent a probability distribution directly. It is rather a language to perform kernel-based learning on expressive logical and relational representations. kLog allows users to specify learning problems declaratively. It builds on simple but powerful concepts: learning from interpretations, entity/relationship data modeling, logic programming, and deductive databases. Access by the kernel to the rich representation is mediated by a technique we call graphicalization: the relational representation is first transformed into a graph — in particular, a grounded entity/relationship diagram. Subsequently, a choice of graph kernel defines the feature space. kLog supports mixed numerical and symbolic data, as well as background knowledge in the form of Prolog or Datalog programs as in inductive logic programming systems. The kLog framework can be applied to tackle the same range of tasks that has made statistical relational learning so popular, including classification, regression, multitask learning, and collective classification. We also report about empirical comparisons, showing that kLog can be either more accurate, or much faster at the same level of accuracy, than Tilde and Alchemy. kLog is GPLv3 licensed and is available at http://klog.dinfo.unifi.it along with tutorials.

💡 Research Summary

The paper introduces kLog, a novel framework for statistical relational learning (SRL) that departs from the traditional approach of directly modeling probability distributions. Instead of defining a probabilistic graphical model, kLog treats relational data as logical interpretations, transforms them into grounded entity‑relationship (ER) graphs, and then applies graph kernels to obtain a high‑dimensional feature space on which standard kernel‑based learners (e.g., SVM, kernel ridge regression) operate. This “learning from interpretations” paradigm preserves the expressive power of first‑order logic while leveraging the computational efficiency of kernel methods.

Key components of kLog are: (1) a declarative specification language that allows users to describe learning tasks, data schemas, and background knowledge in Prolog/Datalog; (2) a graphicalization process that converts each interpretation into a labeled graph. Entities become nodes, relationships become edges, and both symbolic and numeric attributes are encoded as node or edge labels (or auxiliary nodes). Background rules are materialized during this step, enriching the graph with derived facts. (3) a flexible kernel layer. The authors support several well‑known graph kernels—Weisfeiler‑Lehman subtree kernels, shortest‑path kernels, label‑propagation kernels, among others—allowing the practitioner to select a kernel that best captures the structural regularities of the domain. (4) a learning engine that plugs the kernel matrix into any off‑the‑shelf kernel learner, thus avoiding the need for complex parameter estimation typical of Markov Logic Networks (MLNs) or ProbLog.

Because the feature extraction is performed by the kernel, kLog automatically captures high‑order relational patterns without manual feature engineering. Numeric attributes are handled naturally, and symbolic attributes are treated as discrete labels, enabling a seamless mix of data types. Moreover, the system supports multitask learning, regression, classification, and collective classification within the same pipeline; collective dependencies are encoded in the graph structure, so the kernel implicitly accounts for them.

The empirical evaluation compares kLog against two prominent SRL systems: Tilde (an ILP system based on logical regression) and Alchemy (an implementation of Markov Logic Networks). Benchmarks include Mutagenesis (chemical structure classification), WebKB (web page classification), and CiteSeer (citation network analysis). Results show that kLog matches or exceeds the predictive accuracy of the baselines while achieving substantial speedups—often an order of magnitude faster—especially on datasets with many numeric attributes. The authors also report that memory consumption grows with graph size but remains manageable thanks to graph compression techniques.

Strengths highlighted in the paper are: (i) the ability to exploit rich relational structure via graph kernels, (ii) straightforward integration of background knowledge expressed in logic, (iii) compatibility with any kernel learner, and (iv) open‑source availability under GPLv3 with tutorials. Limitations include the potential for high memory usage during graphicalization and the sensitivity of performance to the choice of kernel, which may require domain‑specific tuning.

Future work suggested by the authors includes dynamic graph construction for streaming data, development of sparse or low‑dimensional kernels to reduce memory footprints, hybridization with graph neural networks to learn kernel parameters automatically, and meta‑learning approaches for kernel selection.

In summary, kLog offers a compelling alternative to conventional SRL methods: it retains the declarative, relational expressiveness of logic programming while harnessing the scalability and robustness of kernel‑based learning. By converting logical interpretations into graphs and delegating feature extraction to well‑studied graph kernels, kLog bridges the gap between symbolic relational modeling and statistical learning, providing a versatile tool for both researchers and practitioners in the field of relational machine learning.