Pareto Efficient Multi Objective Optimization for Local Tuning of Analogy Based Estimation

📝 Abstract

Analogy Based Effort Estimation (ABE) is one of the prominent methods for software effort estimation. The fundamental concept of ABE is closer to the mentality of expert estimation but with an automated procedure in which the final estimate is generated by reusing similar historical projects. The main key issue when using ABE is how to adapt the effort of the retrieved nearest neighbors. The adaptation process is an essential part of ABE to generate more successful accurate estimation based on tuning the selected raw solutions, using some adaptation strategy. In this study we show that there are three interrelated decision variables that have great impact on the success of adaptation method: (1) number of nearest analogies (k), (2) optimum feature set needed for adaptation, and (3) adaptation weights. To find the right decision regarding these variables, one need to study all possible combinations and evaluate them individually to select the one that can improve all prediction evaluation measures. The existing evaluation measures usually behave differently, presenting sometimes opposite trends in evaluating prediction methods. This means that changing one decision variable could improve one evaluation measure while it is decreasing the others. Therefore, the main theme of this research is how to come up with best decision variables that improve adaptation strategy and thus, the overall evaluation measures without degrading the others. The impact of these decisions together has not been investigated before, therefore we propose to view the building of adaptation procedure as a multi-objective optimization problem. The Particle Swarm Optimization Algorithm (PSO) is utilized to find the optimum solutions for such decision variables based on optimizing multiple evaluation measures

💡 Analysis

Analogy Based Effort Estimation (ABE) is one of the prominent methods for software effort estimation. The fundamental concept of ABE is closer to the mentality of expert estimation but with an automated procedure in which the final estimate is generated by reusing similar historical projects. The main key issue when using ABE is how to adapt the effort of the retrieved nearest neighbors. The adaptation process is an essential part of ABE to generate more successful accurate estimation based on tuning the selected raw solutions, using some adaptation strategy. In this study we show that there are three interrelated decision variables that have great impact on the success of adaptation method: (1) number of nearest analogies (k), (2) optimum feature set needed for adaptation, and (3) adaptation weights. To find the right decision regarding these variables, one need to study all possible combinations and evaluate them individually to select the one that can improve all prediction evaluation measures. The existing evaluation measures usually behave differently, presenting sometimes opposite trends in evaluating prediction methods. This means that changing one decision variable could improve one evaluation measure while it is decreasing the others. Therefore, the main theme of this research is how to come up with best decision variables that improve adaptation strategy and thus, the overall evaluation measures without degrading the others. The impact of these decisions together has not been investigated before, therefore we propose to view the building of adaptation procedure as a multi-objective optimization problem. The Particle Swarm Optimization Algorithm (PSO) is utilized to find the optimum solutions for such decision variables based on optimizing multiple evaluation measures

📄 Content

1

Pareto Efficient Multi Objective Optimization for Local Tuning of Analogy Based Estimation

Mohammad Azzeh Department of Software Engineering Applied Science University Amman, Jordan POBOX 166 m.y.azzeh@asu.edu.jo Ali Bou Nassif Department of Electrical and
Computer Engineering University of Sharjah Sharjah, UAE abounassif@ieee.org

Shadi Banitaan Department of Mathematics,
Computer Science and Software Engineering University of Detroit Mercy, USA banitash@udmercy.edu

Fadi Almasalha Department of Computer Science Applied Science University Amman, Jordan POBOX 166 f_masalha@asu.edu.jo

Abstract.
Analogy Based Effort Estimation (ABE) is one of the prominent methods for software effort estimation. The fundamental concept of ABE is closer to the mentality of expert estimation but with an automated procedure in which the final estimate is generated by reusing similar historical projects. The main key issue when using ABE is how to adapt the effort of the retrieved nearest neighbors. The adaptation process is an essential part of ABE to generate more successful accurate estimation based on tuning the selected raw solutions, using some adaptation strategy. In this study we show that there are three interrelated decision variables that have great impact on the success of adaptation method: (1) number of nearest analogies (k), (2) optimum feature set needed for adaptation, and (3) adaptation weights. To find the right decision regarding these variables, one need to study all possible combinations and evaluate them individually to select the one that can improve all prediction evaluation measures. The existing evaluation measures usually behave differently, presenting sometimes opposite trends in evaluating prediction methods. This means that changing one decision variable could improve one evaluation measure while it is decreasing the others. Therefore, the main theme of this research is how to come up with best decision variables that improve adaptation strategy and thus, the overall evaluation measures without degrading the others. The impact of these decisions together has not been investigated before, therefore we propose to view the building of adaptation procedure as a multi-objective optimization problem. The Particle Swarm Optimization Algorithm (PSO) is utilized to find the optimum solutions for such decision variables based on optimizing multiple evaluation measures. We evaluated the proposed approaches over 15 datasets and using 4 evaluation measures. After extensive experimentation we found that: (1) predictive performance of ABE has noticeably been improved, (2) optimizing all decision variables together is more efficient than ignoring any one of them. (3) Optimizing decision variables for each project individually yield better accuracy than optimizing them for the whole dataset.

Keywords: Analogy Based Effort Estimation, Adaptation Strategy, Particle Swarm Optimization, Multi-Objective Optimization.

1. Introduction One of the key challenges in software industry is how to obtain the accurate estimation of the development effort, which is particularly important for risk evaluation, resource scheduling as well as progress monitoring [3][15][28]. This importance is clearly portrayed through proposing a vast variety of estimation models in the past years [47]. Inaccuracies in estimations lead to problematic results for both software industry and customers. In one hand the underestimation results in approval of projects that will exceed their planned budgets, while on the other hand the overestimation causes waste of resources and misses opportunities to offer funds for other projects in future [31]. Software effort estimation has been extensively studied in literature since 70’s but they have suffered from common problems such as very large performance deviations as well as being highly dataset dependent [15]. These models can be classified into two main categories: a) Expert Judgment, and b) Learning Oriented models. The former proposes making use of the experiences of human experts whereas, the latter usually generates estimates based on learning methods. The latter has two distinct advantages over the former such that they have capability to model complex set of relationships between dependent variable and the independent variables, and they are capable to learn from historical project data [2][27].
   In the recent years, a significant research effort was put into utilizing various machine learning (ML) algorithms as a complementary or as a replacement to previous methods [14][17][25][31]. Although they generate successful results in certain datasets, ML algorithms suffered from local tuning problems when they 2

were to be applied in another settings, i.e. they need to be tuned to local data for high accuracy values[15][33]. ML methods have an extremely large space of configuration possibilities [15][42][43]. Wh

This content is AI-processed based on ArXiv data.