Coupled human-environment systems are increasingly being understood as complex adaptive systems (CAS), in which micro-level interactions between components lead to emergent behavior. Agent-based models (ABMs) hold great promise for environmental policy design by capturing such complex behavior, enabling a sophisticated understanding of potential interventions. One limitation, however, is that ABMs can be computationally costly to simulate, which hinders their use for policy optimization. To address this, we propose a new statistical framework that exploits machine learning techniques to accelerate policy optimization with costly ABMs. We first develop a statistical approach for sensitivity testing of the optimal policy, then leverage a reinforcement learning method for efficient policy optimization. We test this framework on the classic ``Sugarscape'' model, an ABM for resource harvesting. We show that our approach can quickly identify optimal and interpretable policies that improve upon baseline techniques, with insightful sensitivity and dynamic analyses that connect back to economic theory.
Disorder is at least as old as, and, according to one of the most popular theories, more fundamental than time itself (Rovelli 2018). While the question of to what extent this conception of entropy applies directly to social systems rather than only through physical phenomena is still open (Mavrofides et al. 2011), one notable trend in the social sciences has sought to embrace a certain form of disorder in its approach to this type of research, namely, "complexity." "Complexity" is a conceptually sticky term with no clear, single definition. One common notion of complexity places it at the "edge of chaos," where deterministic systems with strong sensitivity to initial conditions give rise to extremely complicated, hard to parse behavior (see, for example, the figure on page 10 of Strogatz (2018)). Mitchell (2009), however, distinguishes several common properties of such systems and figures them as "system [s] in which large networks of components with no central control and simple rules of operation give rise to complex collective behavior, sophisticated information processing, and adaptation via learning or evolution" (p. 13). 1 The three properties noted at the end of this definition seem broad enough to be almost redundant but, upon further reflection, ground the study of such systems in a very specific tradition of social scientific research, namely computational social science.
Computational social science (CSS) can be broadly thought of as a methodological variation on more traditional approaches to sociological and economic problems, applying computational principles instead of closed-form and equation-based ones. These principles in particular become of use when dealing with complex data-data which does not admit easy unpacking with closed-form equations because of feedbacks, nonlinearities, causal loops, psychological intricacies, etc.-data which is also usually “large-scale” and often simulated in nature (Lazer et al. 2020). If this definition seems opaque, it is largely because the types of methods subsumed under the heading of “computational social science” are diverse and wide-ranging. This paper will focus hereafter almost exclusively on one of these methods, agent-based modeling (ABM).
In particular, this attention to ABMs is warranted by increasing interest in their application to problems of public policy; we here focus on more specific questions of environmental policy, a space within the ABM literature which has undergone greater exploration in recent years but still lacks full methodological clarity. We here hope to contribute to this burgeoning literature by demonstrating both the practicability of applying ABMs to environmental policy questions, and by applying statistically rigorous techniques to the analysis of model outputs, thus solidifying the procedural underpinnings of this research space.
All that said, it would be a gross misrepresentation of prior research to say that no attempts have been made at applying the science of complex systems, in any form including those of agent-based models, to questions of governance and policy. One of the foundational works of computational social science, Epstein and Axtell’s Growing Artificial Societies, notes the policy implications of the agent-based models of price dynamics presented in the work. They cite in particular the power of their non-equilibrium results to challenge conventional wisdom about the operations of markets (Epstein and Axtell 1996). Indeed, it has been the argument of many researchers working on the application of complexity to social systems that it is precisely here where the power of such an approach lies: by allowing for adaptive and dynamic interactions between micro-level actors within the model framework, richer results and more precise policy statements may be made. Existing work on the global sensitivity and uncertainty analysis of ABMs includes Fonoberova et al. (2013); work on statistical inference of parameters has also been done (G. V. Bobashev and R. J. Morris 2010). We provide a more comprehensive literature review of ABMs for policymaking in the next section.
One key bottleneck with ABMs of complex systems, however, is that their simulation can be computationally costly. Such a cost arises from the need to carefully model for complex and fine-scale interactions in the system. Even for the classic “Sugarscape” model (Epstein and Axtell 1996) investigated later in section 5, an ensemble of hundreds to thousands of simulations can take multiple hours to perform. The use of such costly ABMs for policy optimization can thus be computationally very intensive, especially when many policy levers are considered. This cost can hinder the promise of ABMs for policy optimization in practical applications. To address this, we propose a new statistical framework that leverages machine learning techniques to accelerate policy optimization with costly ABMs. In particular, our framework makes use of flexible learning
This content is AI-processed based on open access ArXiv data.