seqme: a Python library for evaluating biological sequence design

Recent advances in computational methods for designing biological sequences have sparked the development of metrics to evaluate these methods performance in terms of the fidelity of the designed sequences to a target distribution and their attainment of desired properties. However, a single software library implementing these metrics was lacking. In this work we introduce seqme, a modular and highly extendable open-source Python library, containing model-agnostic metrics for evaluating computational methods for biological sequence design. seqme considers three groups of metrics: sequence-based, embedding-based, and property-based, and is applicable to a wide range of biological sequences: small molecules, DNA, ncRNA, mRNA, peptides and proteins. The library offers a number of embedding and property models for biological sequences, as well as diagnostics and visualization functions to inspect the results. seqme can be used to evaluate both one-shot and iterative computational design methods.

💡 Research Summary

The paper introduces seqme, an open‑source Python library that consolidates a comprehensive suite of metrics for evaluating computational biological sequence design methods. The authors begin by highlighting a gap in the field: while numerous algorithms now generate DNA, RNA, peptide, protein, and small‑molecule sequences, there is no unified software that implements the diverse set of evaluation criteria that have emerged alongside these methods. To address this, seqme is built around three orthogonal groups of metrics—sequence‑based, embedding‑based, and property‑based—each targeting a different aspect of design quality.

Sequence‑based metrics include classic string similarity measures such as Hamming distance, Levenshtein (edit) distance, k‑mer frequency divergence, and distributional KL‑divergence. These provide a direct, interpretable assessment of how closely generated sequences match a reference set at the nucleotide or amino‑acid level. Embedding‑based metrics leverage pretrained language models (e.g., ESM‑1b, ProtBert, TAPE for proteins; ChemBERTa for small molecules) and structure‑aware embeddings (AlphaFold2, RoseTTAFold). By projecting sequences into high‑dimensional latent spaces, seqme can compute cosine similarity, Euclidean distance, or kernel‑based distribution distances (e.g., Maximum Mean Discrepancy) to capture functional or structural similarity that is invisible to raw‑string comparisons. Property‑based metrics evaluate whether generated sequences achieve desired biophysical or functional characteristics such as solubility, stability, binding affinity, enzymatic activity, or toxicity. The library ships with a collection of pretrained property predictors and offers a plug‑in system for user‑defined models, supporting both regression (MSE, MAE) and classification (ROC‑AUC, F1) performance measures.

From an engineering perspective, seqme adopts a modular architecture centered on three abstract base classes—Metric, Embedding, and Property. Each class defines a standard interface (fit, transform, score) that enables seamless integration of new metrics or models with minimal code changes. Results are returned as pandas DataFrames, facilitating downstream statistical analysis, reporting, and reproducibility. Visualization utilities built on seaborn and matplotlib generate histograms, kernel density estimates, t‑SNE/UMAP plots of embeddings, and scatter plots of property predictions, allowing users to diagnose design failures and compare algorithmic trajectories at a glance.

The library is deliberately agnostic to the design paradigm. For one‑shot generative models (GANs, VAEs, diffusion models) seqme can evaluate the entire output batch in a single call. For iterative optimization schemes (reinforcement learning, Bayesian optimization, evolutionary strategies) it records metric values at each iteration, enabling the construction of learning curves and convergence diagnostics. This uniform API makes it possible to benchmark disparate methods on a common footing, a critical step toward establishing community standards.

Seqme is released under the permissive MIT license on GitHub, with full documentation generated by Sphinx and hosted on ReadTheDocs. Continuous integration pipelines automatically test compatibility across Python versions and common dependency sets. The authors provide Jupyter notebooks that walk users through typical workflows, from loading sequences to computing multi‑metric scores and visualizing results. Community contributions are encouraged through a clear plugin template, coding style guide, and issue‑tracking templates.

Future development plans include extending the property suite to multi‑omics predictors (e.g., metabolomics, proteomics), containerizing the library with Docker and Helm charts for scalable cloud deployment, and exposing a RESTful API for real‑time evaluation in automated design pipelines. By consolidating disparate evaluation tools into a single, extensible framework, seqme aims to improve the rigor, reproducibility, and comparability of biological sequence design research, ultimately accelerating the discovery of functional nucleic‑acid and protein sequences.