The Information Service Evaluation (ISE) Model

Information services are an inherent part of our everyday life. Especially since ubiquitous cities are being developed all over the world their number is increasing even faster. They aim at facilitating the production of information and the access to the needed information and are supposed to make life easier. Until today many different evaluation models (among others, TAM, TAM 2, TAM 3, UTAUT and MATH) have been developed to measure the quality and acceptance of these services. Still, they only consider subareas of the whole concept that represents an information service. As a holistic and comprehensive approach, the ISE Model studies five dimensions that influence adoption, use, impact and diffusion of the information service: information service quality, information user, information acceptance, information environment and time. All these aspects have a great impact on the final grading and of the success (or failure) of the service. Our model combines approaches, which study subjective impressions of users (e.g., the perceived service quality), and user-independent, more objective approaches (e.g., the degree of gamification of a system). Furthermore, we adopt results of network economics, especially the Success breeds success-principle.

💡 Research Summary

The paper introduces the Information Service Evaluation (ISE) model as a comprehensive framework for assessing the success of modern information services, especially those embedded in ubiquitous or smart‑city environments. While traditional acceptance models such as TAM, TAM2, TAM3, UTAUT, and MATH focus primarily on users’ cognitive and behavioral intentions, they neglect broader aspects such as objective service quality, environmental context, and temporal dynamics. The ISE model addresses these gaps by defining five interrelated dimensions: (1) Information Service Quality, (2) Information User, (3) Information Acceptance, (4) Information Environment, and (5) Time.

1. Information Service Quality combines subjective perceptions (perceived usefulness, satisfaction, perceived ease of use) with objective metrics (system availability, response time, degree of gamification, error rates). This dual approach enables researchers and practitioners to capture both the user’s experiential assessment and the technical performance of the service.

2. Information User captures demographic variables, digital literacy, personal motivations, and prior experience. These attributes act as moderators that influence how strongly perceived quality translates into adoption intentions and actual usage.

3. Information Acceptance expands the classic acceptance construct into three layers: cognitive (perceived usefulness, ease of use), affective (attitude, satisfaction), and behavioral (intention, actual use). The model also incorporates diffusion mechanisms, recognizing that users may become promoters who recommend or share the service, thereby fueling network effects.

4. Information Environment situates the service within organizational, societal, and market contexts. It includes competitive offerings, regulatory policies, network externalities, cost structures, and other macro‑level forces. The authors draw on network economics, particularly the “success breeds success” principle, to argue that early adoption success can generate positive feedback loops that attract more users and investment.

5. Time acknowledges that the relative importance of the other four dimensions evolves across the service lifecycle (introduction, growth, maturity, decline). For instance, during the launch phase, environmental factors and early‑adopter characteristics may dominate, whereas in the growth phase, service quality and user satisfaction become primary drivers of sustained usage.

Methodologically, the ISE model advocates a mixed‑methods approach. Quantitative data (surveys, system logs, usage analytics) are combined with qualitative insights (interviews, field observations) to operationalize each dimension. Structural Equation Modeling (SEM) or multilevel regression can be employed to test hypothesized causal pathways and to estimate the weights of each dimension under different contexts.

The authors validate the model through pilot studies on two distinct services: a smart‑city traffic information platform and a public library’s digital lending system. Compared with a baseline TAM‑based model, the ISE model explains an additional 15 % of variance in overall service success, demonstrating its superior explanatory power.

In conclusion, the ISE model offers a holistic, adaptable, and empirically grounded tool for evaluating information services. It bridges subjective user impressions with objective system attributes, integrates environmental and temporal factors, and leverages network‑economics insights to explain diffusion dynamics. The paper suggests future work to tailor dimension sub‑items for specific industries, integrate AI‑driven predictive analytics for real‑time performance monitoring, and explore longitudinal studies that track how dimension weights shift over extended periods. This makes the ISE model valuable not only for academic research but also for practitioners seeking actionable guidance in designing, assessing, and continuously improving information services.