Enhancing Psychometric Analysis with Interactive SIA Modules

ShinyItemAnalysis (SIA) is an R package and shiny application for an interactive presentation of psychometric methods and analysis of multi-item measurements in psychology, education, and social sciences in general. In this article, we present a new feature introduced in the recent version of the package, called “SIA modules”, which allows researchers and practitioners to offer new analytical methods for broader use via add-on extensions. SIA modules are designed to integrate with and build upon the SIA interactive application, enabling them to leverage the existing infrastructure for tasks such as data uploading and processing. They can access and further use a range of outputs from various analyses, including models and datasets. Because SIA modules come in R packages (or extend the existing ones), they may come bundled with their datasets, use object-oriented systems, or even compiled code. We illustrate the concepts using sample modules from the newly introduced SIAmodules package and other packages. After providing a general overview of building Shiny applications, we describe how to develop the SIA add-on modules with the support of the new SIAtools package. Finally, we discuss possibilities of future development and emphasize the importance of freely available, interactive psychometric software for dissemination of methodological innovations.

💡 Research Summary

The paper introduces “SIA modules,” a new extensibility framework for the ShinyItemAnalysis (SIA) package, which is an R‑based Shiny application for interactive psychometric analysis. Traditional SIA already offers a comprehensive suite of classical test theory (CTT), item response theory (IRT), and differential item functioning (DIF) tools through both a command‑line interface and a graphical web interface. However, adding novel methods or complex models required programming expertise, limiting accessibility for non‑technical users.

SIA modules solve this problem by packaging additional analytical capabilities as separate R packages that integrate seamlessly with the core SIA application via the newly created SIAtools package. The architecture follows three main principles: (1) reuse of SIA’s existing infrastructure for data upload, preprocessing, and visualization; (2) sharing of reactive objects and output tables so that modules can react instantly to changes in the main app; and (3) encapsulation of UI and server logic into independent modules, enabling straightforward development, distribution, and runtime loading.

Three illustrative modules are presented. The EduTest Item Analysis module (in the EduTestItemAnalysis package) demonstrates custom data upload for Czech Matura exam data, supporting heterogeneous item types (3PL, 2PL, GPCM, NRM) within a single test. It provides a “Pass data to SIA” button that transfers edited data back to the main application, allowing downstream DIF analysis and other procedures. The Computerized Adaptive Testing (CAT) module (part of the SIAmodules package) simulates an adaptive test: users select an IRT model and a respondent ability, the module generates responses, selects the most informative item at each step, updates the ability estimate, and visualizes information functions and standard errors. Crucially, the CAT module can either use a model fitted in the main SIA interface or define its own, illustrating cross‑module model sharing. The DIF‑C module extends DIF analysis to a longitudinal setting by incorporating pre‑test scores as matching variables in a regression‑based DIF framework, enabling detection of “DIF in change.” This is shown with a study on learning competencies where, after controlling for 6th‑grade scores, certain 9th‑grade items still displayed differential functioning across school tracks.

A comparative table positions SIA against other interactive psychometric tools such as jMetrik, WINSTEPS, IRTPRO, Mplus, and flexMIRT. SIA stands out for being fully open‑source, offering complete code‑level customization, and supporting modular extensions through a unified R environment. The paper also details the user experience: when ShinyItemAnalysis::run_app() is launched, it checks for missing dependencies, offers to install available modules from the official SIA repository, and provides a “Rediscover modules” button in the GUI to load newly installed packages without restarting the app.

Future directions include adding structural equation modeling, multidimensional IRT, and text‑analysis capabilities as separate modules, as well as establishing a community‑driven repository where researchers can contribute their own extensions. By lowering the barrier to advanced psychometric methods, SIA modules aim to broaden the reach of rigorous measurement practices across education, testing, and health research, while promoting reproducibility and collaborative development.