Practitioners Perspectives on Change Impact Analysis for Safety-Critical Software - A Preliminary Analysis

📝 Abstract

Safety standards prescribe change impact analysis (CIA) during evolution of safety-critical software systems. Although CIA is a fundamental activity, there is a lack of empirical studies about how it is performed in practice. We present a case study on CIA in the context of an evolving automation system, based on 14 interviews in Sweden and India. Our analysis suggests that engineers on average spend 50-100 hours on CIA per year, but the effort varies considerably with the phases of projects. Also, the respondents presented different connotations to CIA and perceived the importance of CIA differently. We report the most pressing CIA challenges, and several ideas on how to support future CIA. However, we show that measuring the effect of such improvement solutions is non-trivial, as CIA is intertwined with other development activities. While this paper only reports preliminary results, our work contributes empirical insights into practical CIA.

💡 Analysis

Safety standards prescribe change impact analysis (CIA) during evolution of safety-critical software systems. Although CIA is a fundamental activity, there is a lack of empirical studies about how it is performed in practice. We present a case study on CIA in the context of an evolving automation system, based on 14 interviews in Sweden and India. Our analysis suggests that engineers on average spend 50-100 hours on CIA per year, but the effort varies considerably with the phases of projects. Also, the respondents presented different connotations to CIA and perceived the importance of CIA differently. We report the most pressing CIA challenges, and several ideas on how to support future CIA. However, we show that measuring the effect of such improvement solutions is non-trivial, as CIA is intertwined with other development activities. While this paper only reports preliminary results, our work contributes empirical insights into practical CIA.

📄 Content

Practitioners’ Perspectives on Change Impact Analysis for Safety-Critical Software – A Preliminary Analysis Markus Borg1, José-Luis de la Vara2, and Krzysztof Wnuk3

1 SICS Swedish ICT AB, Lund, Sweden markus.borg@sics.se 2 Carlos III University of Madrid, Spain jvara@inf.uc3m.es 3 Blekinge Institute of Technology, Karlskrona, Sweden krzysztof.wnuk@bth.se Abstract. Safety standards prescribe change impact analysis (CIA) during evo- lution of safety-critical software systems. Although CIA is a fundamental ac- tivity, there is a lack of empirical studies about how it is performed in practice. We present a case study on CIA in the context of an evolving automation sys- tem, based on 14 interviews in Sweden and India. Our analysis suggests that engineers on average spend 50-100 hours on CIA per year, but the effort varies considerably with the phases of projects. Also, the respondents presented different connotations to CIA and perceived the importance of CIA differently. We report the most pressing CIA challenges, and several ideas on how to support future CIA. However, we show that measuring the effect of such improvement solutions is non-trivial, as CIA is intertwined with other devel- opment activities. While this paper only reports preliminary results, our work contributes empirical insights into practical CIA. Keywords: change impact analysis, safety-critical systems, case study research. 1 Introduction Safety-critical software systems evolve during their lifecycle. As changes are made to the systems, change impact analysis (CIA) is needed, defined as “identifying the po- tential consequences of a change in a system, or estimating what needs to be modified to accomplish a change” [2]. CIA is essential for safety assurance, and it is indeed prescribed by safety standards, e.g. IEC 61508 states that “if at any phase of the soft- ware safety lifecycle, a modification is required pertaining to an earlier lifecycle phase, then an impact analysis shall determine (1) which software modules are im- pacted, and (2) which earlier safety lifecycle activities shall be repeated.” CIA is often a difficult task in practice due to the size and complexity of safety- critical systems [2, 7]. Inadequate CIA has further been among the causes of accidents and near-accidents in the past [10]. Industry can clearly benefit from new CIA tech- nology and knowledge to more cost-effectively perform this safety assurance activity, enabling better risk avoidance and mitigation. Such technology and knowledge must be linked to current practices and targeted at meeting industry needs and expectations.
Despite the importance of CIA for safety-critical systems, the current knowledge about the state of the practice is limited. We are not aware of any publication that has studied the CIA activity in depth. The available knowledge is based on studies that (1) have dealt with non-safety-critical systems, (2) have analyzed data from past projects, (3) have not focused on CIA, or (4) have surveyed practices for safety-critical systems from a general perspective. For example, Rovegård et al. [14] interviewed software practitioners to analyze CIA issue importance, whereas Borg et al. [3] studied past issue reports of an industrial control system. Some insights have been provided in studies on e.g. the alignment of requirements with verification and validation [1] and on traceability [13]. Regarding the surveys, Nair et al. [12] studied safety evidence management practices, including certain aspects related to change management, and de la Vara et al. [7] conducted a survey on safety evidence CIA to explore the circum- stances under which it is performed, the tool support used, and the challenges faced. We have conducted an industrial case study on CIA for safety-critical systems in practice, particularly exploring engineers’ views on the work involved. The context is a distributed development organization offering industrial control systems to a global market. We interviewed 14 engineers in two units of analyses, constituted of two teams located in Sweden and India, respectively. This paper reports a preliminary analysis covering a subset of the interview guide. Our long term goal is to support architectural decision making when evolving cyber-physical systems, an endeavor in which the CIA is fundamental. As a step in this direction, we explore three research questions: RQ1) How extensive is the CIA work task?, RQ2) What are the engineers’ attitudes toward CIA?, and RQ3) How could CIA be supported? By better understanding CIA in a particular case, we can take steps toward understanding how previous knowledge could be stored to support decision making in software evolution, in line with our previous work on traceability reuse [3] and knowledge repositories [6]. The rest of the paper is structured as follows: Section 2 presents the case, and Sec- tion 3 describes the research methodology. We

This content is AI-processed based on ArXiv data.