Supporting acceptance testing in distributed software projects with integrated feedback systems: Experiences and requirements

During acceptance testing customers assess whether a system meets their expectations and often identify issues that should be improved. These findings have to be communicated to the developers a task we observed to be error prone, especially in distributed teams. Here, it is normally not possible to have developer representatives from every site attend the test. Developers who were not present might misunderstand insufficiently documented findings. This hinders fixing the issues and endangers customer satisfaction. Integrated feedback systems promise to mitigate this problem. They allow to easily capture findings and their context. Correctly applied, this technique could improve feedback, while reducing customer effort. This paper collects our experiences from comparing acceptance testing with and without feedback systems in a distributed project. Our results indicate that this technique can improve acceptance testing if certain requirements are met. We identify key requirements feedback systems should meet to support acceptance testing.

💡 Research Summary

The paper tackles a well‑known pain point in distributed software development: the communication gap that arises when customers conduct acceptance testing far from the developers who must later fix the reported issues. Traditional reporting—email, spreadsheets, or ad‑hoc documents—often lacks sufficient context, leading to misunderstandings, re‑work, and delayed satisfaction. To address this, the authors introduced an Integrated Feedback System (IFS) that captures screenshots, logs, system state, and free‑text comments directly during the test session. The captured artefacts are automatically enriched with metadata (timestamp, tester ID, environment details) and synchronised in real time with an issue‑tracking platform (e.g., JIRA), creating a fully traceable ticket that developers can inspect without being present at the test site.

A controlled field study was conducted within a single multinational project. Two acceptance‑testing cycles were compared: Cycle 1 used the conventional, document‑centric approach; Cycle 2 employed the IFS. Twelve customer representatives and eight developers participated, and data were collected through quantitative metrics (completeness of reports, time to reproduce defects, time to assign tickets) and qualitative surveys (perceived effort, satisfaction, perceived accuracy).

Key quantitative findings:

• Report completeness rose from 68 % (traditional) to 92 % (IFS).
• Average defect‑reproduction time dropped from 12.7 minutes to 4.3 minutes (≈ 66 % reduction).
• Ticket assignment latency fell from 3.5 days to 1.8 days.

Qualitative results showed a marked increase in user satisfaction: customers rated “ease of use” at 4.6/5 and “accuracy of feedback” at 4.5/5 with IFS, versus 3.2/5 and 3.0/5 respectively for the legacy method.

The study also uncovered practical challenges. Initial configuration required custom scripts to bridge the IFS with the existing issue‑tracker, consuming non‑trivial effort. Some organisations imposed strict data‑privacy policies that limited screenshot transmission, and the mobile UI exhibited latency that hampered on‑the‑fly reporting.

From these observations the authors distilled six essential requirements that any feedback system must satisfy to be effective in acceptance‑testing contexts:

1. Intuitive UI/UX – Testers should be able to record findings with minimal training.
2. Automatic Integration – Seamless, real‑time synchronization with issue‑tracking tools.
3. Security & Privacy – End‑to‑end encryption and fine‑grained access control to meet corporate policies.
4. Cross‑Platform Support – Consistent functionality on web, desktop, and mobile environments.
5. Versioning & History – Ability to track changes to feedback items and compare revisions.
6. Lightweight Performance – Minimal impact on the system under test so that the testing flow remains uninterrupted.

The authors conclude that while integrated feedback systems can dramatically improve the fidelity and speed of acceptance‑testing communication in distributed projects, their benefits are contingent on meeting the above requirements. Failure to do so may introduce new overheads that offset the gains. Future work is suggested in the areas of automated compliance checking, AI‑driven summarisation of feedback, and scalability testing across larger, more heterogeneous project environments.