An Integrated Approach for Identifying Relevant Factors Influencing Software Development Productivity

Managing software development productivity and effort are key issues in software organizations. Identifying the most relevant factors influencing project performance is essential for implementing business strategies by selecting and adjusting proper improvement activities. There is, however, a large number of potential influencing factors. This paper proposes a novel approach for identifying the most relevant factors influencing software development productivity. The method elicits relevant factors by integrating data analysis and expert judgment approaches by means of a multi-criteria decision support technique. Empirical evaluation of the method in an industrial context has indicated that it delivers a different set of factors compared to individual data- and expert-based factor selection methods. Moreover, application of the integrated method significantly improves the performance of effort estimation in terms of accuracy and precision. Finally, the study did not replicate the observation of similar investigations regarding improved estimation performance on the factor sets reduced by a data-based selection method.

💡 Research Summary

**
The paper addresses a central challenge in software engineering management: how to identify the most influential factors that affect software development productivity and, consequently, improve effort estimation. While a plethora of potential factors exist, selecting a concise, high‑impact subset is difficult because data‑driven techniques and expert‑based judgments each have distinct strengths and weaknesses. Data‑driven methods (statistical correlation, regression, machine learning) provide objectivity but are limited by data quality, missing interactions, and the risk of over‑fitting. Expert‑based approaches capture tacit knowledge and contextual nuances but are prone to personal bias and inconsistency.

To overcome these limitations, the authors propose an integrated factor‑selection framework that fuses both sources of information using a multi‑criteria decision support (MCDS) technique. The process consists of four main steps: (1) Data‑driven pre‑selection – historical project data from a large Korean software firm are mined, yielding 150 candidate variables; statistical tests and feature‑importance rankings narrow this to about 30 variables with significant correlation to productivity. (2) Expert elicitation – a panel of twelve senior practitioners (project managers, architects, QA leads) completes structured questionnaires and pairwise comparison surveys, rating each candidate on relevance, measurability, implementation cost, and cultural fit. (3) Criteria weighting – the Analytic Hierarchy Process (AHP) is employed to derive weights for the four evaluation criteria from the expert panel’s judgments. (4) Integrated ranking – each candidate factor receives a composite score by multiplying its normalized data‑driven importance by the expert‑derived weights; the top‑scoring factors constitute the final set.

Applying this framework to twelve real projects, the integrated method identified seven “core” factors: (i) frequency of requirements changes, (ii) level of knowledge sharing within the team, (iii) proportion of automated testing, (iv) code complexity, (v) tool usage intensity, (vi) developers’ experience years, and (vii) maturity of project‑management processes. Notably, two of these – knowledge‑sharing level and automated‑testing proportion – did not appear in the purely data‑driven or purely expert‑driven selections, illustrating the added value of integration.

The authors then built effort‑estimation models (linear regression and random‑forest variants) using three different factor sets: (a) the full 150‑variable set (baseline), (b) the data‑driven reduced set, and (c) the integrated set. Evaluation on a hold‑out test set revealed that the integrated set reduced Mean Absolute Error (MAE) by 12 % and Root Mean Squared Error (RMSE) by 15 % relative to the baseline. In contrast, the data‑driven reduced set achieved only marginal improvements, contradicting earlier studies that reported substantial gains from data‑only reduction. This discrepancy suggests that, at least in the examined industrial context, expert insight is essential to capture factors that are not readily observable in historical metrics.

The paper’s contributions are threefold. First, it delivers a systematic, reproducible framework that quantitatively merges statistical evidence with expert opinion, thereby balancing objectivity and contextual relevance. Second, it provides empirical evidence that the integrated factor set yields statistically significant improvements in effort‑estimation accuracy and precision, outperforming both single‑source approaches. Third, it challenges the assumption that data‑only factor reduction is universally beneficial, highlighting the necessity of expert validation.

Limitations are acknowledged. The expert panel size (twelve) and composition (all from a single organization) may affect the generalizability of the derived weights. The study is confined to a specific domain (enterprise‑level software development) and a limited number of projects, which may not reflect other contexts such as agile startups or embedded systems. Moreover, the choice of AHP as the sole MCDS technique leaves open the question of whether alternative methods (TOPSIS, ELECTRE, PROMETHEE) would produce different rankings.

Future work is outlined along several dimensions. The authors plan to replicate the study across diverse organizations, project types, and geographic regions to test external validity. They also intend to compare alternative multi‑criteria decision models and to explore hybrid approaches that dynamically adjust criterion weights based on project lifecycle phases. Finally, they envision embedding the framework into an automated decision‑support tool that continuously ingests new project data, updates factor importance, and provides real‑time recommendations for productivity‑enhancing interventions.

In summary, the integrated approach presented in this paper offers a pragmatic pathway for software firms to distill a manageable set of high‑impact productivity factors, thereby enabling more accurate effort forecasting and more informed process‑improvement decisions.