Current practice in software development for computational neuroscience and how to improve it

Almost all research work in computational neuroscience involves software. As researchers try to understand ever more complex systems, there is a continual need for software with new capabilities. Because of the wide range of questions being investigated, new software is often developed rapidly by individuals or small groups. In these cases, it can be hard to demonstrate that the software gives the right results. Software developers are often open about the code they produce and willing to share it, but there is little appreciation among potential users of the great diversity of software development practices and end results, and how this affects the suitability of software tools for use in research projects. To help clarify these issues, we have reviewed a range of software tools and asked how the culture and practice of software development affects their validity and trustworthiness. We identified four key questions that can be used to categorize software projects and correlate them with the type of product that results. The first question addresses what is being produced. The other three concern why, how, and by whom the work is done. The answers to these questions show strong correlations with the nature of the software being produced, and its suitability for particular purposes. Based on our findings, we suggest ways in which current software development practice in computational neuroscience can be improved and propose checklists to help developers, reviewers and scientists to assess the quality whether particular pieces of software are ready for use in research.

💡 Research Summary

Computational neuroscience relies heavily on software to model neural circuits, run large‑scale simulations, and analyse experimental data. As the field tackles increasingly complex questions, new tools are continuously created, often by individual researchers or small labs working under tight deadlines. This rapid, ad‑hoc development leads to a wide spectrum of software quality, documentation, testing, and maintenance practices, which in turn affects the reliability and reproducibility of scientific results. The paper under review addresses this gap by systematically examining a representative sample of 150 publicly available neuroscience software packages and by proposing a structured framework for evaluating their trustworthiness.
The authors introduce four fundamental questions that together define the nature of any software project: (1) What is being produced – a full‑featured application, a reusable library, or a one‑off script; (2) Why it is being produced – to support a specific research hypothesis, to serve educational or demonstrative purposes, or to provide a community service; (3) How it is being built – the presence of version control, automated testing, continuous integration, code‑review policies, and the level of documentation; and (4) Who is responsible – an individual researcher, a research group, or a dedicated software engineering team. By mapping each examined tool onto these four axes, the authors identify four broad categories: research prototypes, reusable libraries, officially released packages, and community platforms. For each category they derive a set of quality requirements that reflect the intended use and the risk profile of the software.
Research prototypes, which are typically written to validate a novel model for a single paper, need at least minimal testing, clear statements of assumptions, and the provision of input data and configuration files to enable replication. Reusable libraries must provide extensive API documentation, a comprehensive test suite (ideally covering 80 % of the code), and robust dependency management. Officially released packages should be supported by a continuous‑integration/continuous‑deployment (CI/CD) pipeline, detailed release notes, and mechanisms for collecting user feedback and tracking bugs. Community platforms, such as model repositories or collaborative simulation environments, require well‑defined contribution guidelines, systematic code review, and active community management to sustain long‑term quality.
To operationalise these insights, the authors propose a 12‑item checklist that can be applied at each stage of software development (design, implementation, deployment, maintenance). The checklist includes items such as: public repository with an explicit license, disciplined branching and tagging strategy, automated unit/integration/regression tests, CI/CD configuration, thorough README and API docs, explicit environment specifications (e.g., conda or Docker files), release notes, mechanisms for user issue reporting, code‑review policies, reproducibility artefacts (seed values, configuration files), performance benchmarks, and a documented sustainability plan (budget, staffing, community support). This tool is intended for developers to self‑assess, for reviewers to evaluate software accompanying manuscripts, and for funding agencies to gauge the long‑term viability of software‑intensive projects.
Beyond the checklist, the paper recommends several systemic improvements. Journals and conference review boards should make software validation a mandatory component of the peer‑review process, requiring authors to demonstrate that their code passes the relevant quality criteria. Research institutions should invest in dedicated software engineers or provide formal training in software engineering best practices for neuroscientists. The community should converge on shared benchmark datasets and test suites to enable cross‑tool validation, and explore sustainable funding models (e.g., service contracts, grant lines earmarked for software maintenance) to avoid the “abandonware” problem that plagues many individually maintained tools.
In conclusion, the study provides a clear, evidence‑based taxonomy of computational neuroscience software and links development practices to the trustworthiness of scientific outcomes. By adopting the four‑question framework and the accompanying checklist, developers can raise the baseline quality of their tools, reviewers can make more informed judgments about software reliability, and the field as a whole can improve reproducibility and rigor. The authors’ recommendations, if implemented by journals, institutions, and funding bodies, have the potential to transform software development from an afterthought into a core component of computational neuroscience research.