Background: Philosophers of science including Collins, Feyerabend, Kuhn and Latour have all emphasized the importance of consensus within scientific communities of practice. Consensus is important for maintaining legitimacy with outsiders, orchestrating future research, developing educational curricula and agreeing industry standards. Low consensus contrastingly undermines a field's reputation and hinders peer review. Aim: This paper aims to investigate the degree of consensus within the software engineering academic community concerning members' implicit theories of software engineering. Method: A convenience sample of 60 software engineering researchers produced diagrams describing their personal understanding of causal relationships between core software engineering constructs. The diagrams were then analyzed for patterns and clusters. Results: At least three schools of thought may be forming; however, their interpretation is unclear since they do not correspond to known divisions within the community (e.g. Agile vs. Plan-Driven methods). Furthermore, over one third of participants do not belong to any cluster. Conclusion: Although low consensus is common in social sciences, the rapid pace of innovation observed in software engineering suggests that high consensus is achievable given renewed commitment to empiricism and evidence-based practice.
Scientific consensus simply refers to the level of general and widespread (but not perfect or unanimous) agreement within a particular scientific community on key topics. Scientific progress is exceptionally difficult in fields that lack consensus. Without consensus, all facts seem equally relevant and "different men confronting the same range of phenomena … describe and interpret them in different ways" [21]. Rather than incrementally improving good theories and rejecting bad theories, academics in low consensus fields talk past each other with successive, disconnected results. Lack of consensus therefore inhibits the development of a cumulative body of knowledge, undermines the reliability of peer review, and delegitimizes the field to outsiders. Educational curricula and industry standards cannot reflect the non-existent consensus.
When consensus is high, however, the scope of the field narrows and fewer types of investigations are perceived as meaningful. The ensuing development of specific norms, specialized equipment and advanced skills highlights previously obscured anomalies, which trigger the next paradigm shift [21]. The pace of innovation increases and more consensus develops as the advancing discovery frontier eclipses bickering over yesterday’s disagreements [10]. Scientists focus on extending one another’s work rather than arguing.
Meanwhile, much software engineering (SE) research aims to develop technologies and practices that will help developers to succeed in their SE initiatives. However, due to the complexity of software development in practice, demonstrating a direct causal link between a specific artifact and overall performance is extraordinarily difficult. Empirical researchers therefore focus on intermediate variables including quality of specifications, financial risk and developer motivation. How these intermediate variables interact, however, remains unclear.
Here, consensus is important regardless of its correctness. Problems with clear, widely-accepted theories may be rapidly revealed by empirical research. Clear, consensus theory may therefore be quickly improved or rejected. Unclear theory or general dissensus, contrastingly, cannot be easily tested and rapidly improved. As Francis Bacon famously said, “Truth emerges more readily from error than from confusion.” Moreover, Kuhn [21] explains that scientific communities mature in three stages -1) in the “no science at all” stage no common ground exists between the scholars in a discipline; 2) in the “prescience” stage, a limited number of competing schools hold contradicting theories; 3) in the “normal science” stage, the vast majority adhere to a single theoretical base. This motivates the following research question.
Research Question: What (Kuhnian) stage of consensus best describes the software engineering academic community, specifically concerning causal relationships between core constructs?
Here, a core construct is a variable that is often hypothesized to affect software engineering outcomes, or is commonly associated with SE performance. Core constructs may be positively or negatively related to outcomes or each other. Moreover, a core construct is something popularly believed to influence SE outcomes -whether or not this belief is not correct. More generally, this paper does not address correctness or accuracy in any sense; rather, we focus on the similarities and differences of SE academics’ belief structures.
We not turn to the study’s empirical methodology ( §2), findings ( §3) and their interpretation ( §4). Next, we discuss related research on scientific consensus (our motivation), causal graphs (our data), previous attempts to generate success theories and how our results relate to general theory in software engineering ( §5). Section 6 concludes the paper by summarizing its contributions, limitations and implications for future research.
The general idea of the study is to have SE academics diagram their mental models of relationships among core SE constructs and then compare the diagrams to establish the extent and nature of consensus. This section describes the pre-study used to generate the list of constructs, population and sampling, the diagramming task, hypotheses, and the analytical approach.
Practically speaking, it seems unlikely that even SE experts would be capable of generating a comprehensive list of variables quickly and without forethought. We therefore decided to provide a list of variables from which to choose. We developed this list using the following procedure.
including the term software engineering) which contain any of the following terms: improve, effect, impact, benefits, increase, effects, achieve, affect, factors, increased, causes, benefit, impacts, influence, affects, cause, caused, influences, lead to, contribute, improves, depends on, increases, determines, contributes, determine, result in, function of, leads to, gain, decrease, gains, accomplish, variables.
- For eac
This content is AI-processed based on open access ArXiv data.
