This paper reviews the role of expert judgement to support reliability assessments within the systems engineering design process. Generic design processes are described to give the context and a discussion is given about the nature of the reliability assessments required in the different systems engineering phases. It is argued that, as far as meeting reliability requirements is concerned, the whole design process is more akin to a statistical control process than to a straightforward statistical problem of assessing an unknown distribution. This leads to features of the expert judgement problem in the design context which are substantially different from those seen, for example, in risk assessment. In particular, the role of experts in problem structuring and in developing failure mitigation options is much more prominent, and there is a need to take into account the reliability potential for future mitigation measures downstream in the system life cycle. An overview is given of the stakeholders typically involved in large scale systems engineering design projects, and this is used to argue the need for methods that expose potential judgemental biases in order to generate analyses that can be said to provide rational consensus about uncertainties. Finally, a number of key points are developed with the aim of moving toward a framework that provides a holistic method for tracking reliability assessment through the design process.
Statistics is considered one of the major contributors to the development of reliability engineering as a technical discipline [131]. Recent reviews of the role of statistics within reliability engineering [12,82,92,102] underline the continued need for statistical science to help engineers assess sources of uncertainty, design sound data collection systems, and develop models for combining data and quantifying uncertainty. However it is also recognized that the role of statistical science within the engineering process needs to broaden to accommodate the additional complexities of the technological systems as well as the operational contexts. One particular challenge is the need to structure and integrate statistical modeling within the systems engineering process to support decision-making aimed at obtaining a sufficient and cost effective state of knowledge about future system reliability. This implies that judgemental, as well as objective, data should be collected responsibly and used formally.
This paper aims to survey and review the use of subjective expert judgement methods to assess reliability in the design process. We have deliberately chosen to interpret the scope of these terms in a relatively broad fashion. Thus “expert judgement” refers to any structured method of acquiring knowledge from experts; “reliability” covers the broader issues of reliability, availability and maintainability (RAM); and the “design process” is considered to include within its scope a consideration of how the system is to be manufactured, how users will interact with it and how it will be maintained. More specifically, since reliability measures are usually expressed in probabilistic terms, we consider the use of expert judgement to structure probabilistic models and to quantify uncertainties in the development of a reliable design.
The standard definition of reliability, “the ability of a system to perform a required function under stated conditions for a stated period of time” [70], naturally translates into a probability measure. While empirical reliability can only be properly assessed after a system is in use, there is a need to forecast reliability during the design process to support analysis aimed at improving reliability. Davis [33] supported the definition found in [23] that “reliability is failure mode avoidance.” We are sympathetic to this view since identifying and mitigating influential critical failure modes will cause reliability to improve. However, we also believe that probabilistic models have an important role to play in supporting design decisions since they allow data integration and assist prioritization.
Reliability is a recognized element of systems engineering and systems design. However, it is worth recognizing from the outset how difficult it is to talk about the reliability of a system. In part the difficulty has to do with ambiguity of any reliability metric. In modern systems engineering the practice of requirements setting should, if carried out well, result in a coherent set of reliability requirements expressed in terms of well-defined RAM metrics. Hence good engineering-management practice should ensure that there is little ambiguity in the expression of reliability requirements. More difficult though is the uncertainty around the circumstances under which those requirements are to be met. The reliability of a system is ultimately determined by a combination of factors. Simplistically, we may think of the reliability of a specific system as being determined by the detailed design reliability as modulated by induced unreliabilities coming from the manufacturing process, from the users, from maintenance and from modifications. Simplistically, detailed design reliability gives the maximum potential reliability which manufacturing errors, poor usage and poor maintenance will typically reduce, while changes or modifications introduced as a result of experience with the equipment will improve the reliability, that is overall reliability = designed reliability production unreliability usage unreliability maintenance unreliability + changes reliability.
More compactly, we could write a chosen reliability measure r as r = r(d, p, u, m, c), where d, p, u, m and c represent the choices made for design, production, usage, maintenance and changes. Inasmuch as systems engineering is about making trade-offs between different aspects of the system, the major focus for expert judgement techniques in support of reliability has to be to explore the behavior of, and even quantify, the above conceptual function in some way.
The existing expert judgement literature is a starting point for elicitation problems in engineering design, but it needs to be extended to cope with the unique problems encountered. This is one of the motivations for the present paper. In discussing the ways in which expert judgement methods are adopted to assess uncertainties in the design process we shall consider both the aca
This content is AI-processed based on open access ArXiv data.