Machine learning is a computational process. To that end, it is inextricably tied to computational power - the tangible material of chips and semiconductors that the algorithms of machine intelligence operate on. Most obviously, computational power and computing architectures shape the speed of training and inference in machine learning, and therefore influence the rate of progress in the technology. But, these relationships are more nuanced than that: hardware shapes the methods used by researchers and engineers in the design and development of machine learning models. Characteristics such as the power consumption of chips also define where and how machine learning can be used in the real world. Despite this, many analyses of the social impact of the current wave of progress in AI have not substantively brought the dimension of hardware into their accounts. While a common trope in both the popular press and scholarly literature is to highlight the massive increase in computational power that has enabled the recent breakthroughs in machine learning, the analysis frequently goes no further than this observation around magnitude. This paper aims to dig more deeply into the relationship between computational power and the development of machine learning. Specifically, it examines how changes in computing architectures, machine learning methodologies, and supply chains might influence the future of AI. In doing so, it seeks to trace a set of specific relationships between this underlying hardware layer and the broader social impacts and risks around AI.
Machine learning is a computational process. To that end, it is inextricably tied to computational power -the tangible material of chips and semiconductors that the algorithms of machine intelligence operate on. Most obviously, computational power and computing architectures shape the speed of training and inference in machine learning, and therefore influence the rate of progress in the technology. But, these relationships are more nuanced than that: hardware shapes the methods used by researchers and engineers in the design and development of machine learning models. Characteristics such as the power consumption of chips also define where and how machine learning can be used in the real world.
In a broader perspective, computational power is also important because of its specific geographies. Semiconductors are designed, fabricated, and deployed through a complex international supply chain. Market structure and competition among companies in this space influence the progress of machine learning. Moreover, since these supply chains are also considered significant from a national security perspective, hardware becomes an arena in which government industrial and trade policy has a direct impact on the fundamental machinery necessary for artificial intelligence (AI). Despite this, many analyses of the social impact of the current wave of progress in AI have not substantively brought the dimension of hardware into their accounts. While a common trope in both the popular press and scholarly literature is to highlight the massive increase in computational power that has enabled the recent breakthroughs in machine learning, the analysis frequently goes no further than this observation around magnitude. This paper aims to dig more deeply into the relationship between computational power and the development of machine learning. Specifically, it examines how changes in computing architectures, machine learning methodologies, and supply chains might influence the future of AI. In doing so, it seeks to trace a set of specific relationships between this underlying hardware layer and the broader social impacts and risks around AI. On one hand, this examination shines a spotlight on how hardware works to exacerbate a range of concerns around ubiquitous surveillance, technological unemployment, and geopolitical conflict. On the other, it also highlights the potentially significant role that shaping the development of computing power might play in addressing these concerns. Part I will examine the role that computational power has played in the progress of machine learning, arguing that its impact has been somewhat flattened in recent accounts looking at the social impact of the technology. Part II will look at trends towards increasing specialization in the hardware used for machine learning, and its implications for control and privacy in the space. Part III will look at the semiconductor supply chain, and its implications for the geopolitics of machine learning. Part IV will examine research developments changing the balance between data and computational power in the workflow of machine learning, and its influence on the economic impact of the technology. It will then conclude with some remarks on the potential role of hardware as a lever for policy action in the space.
AI has historically moved through multiple cycles of progress and optimism followed by setbacks and pessimism, so called “AI winters”. 2 Present-day excitement around AI, and more specifically the recent breakthroughs in the subfield of machine learning, represent only the latest upswing in this historical pattern.
Machine learning itself, the study of algorithms which improve themselves through data, is not a new domain of research. The fundamentals underlying the modern advances in the field were established by researchers in the 1950s and developed throughout the subsequent decades. 3 However, neural networks -the specific technique of machine learning driving much of the commercial interest in AI today -were still considered a niche area of research only until relatively recently. As one popular account has put it, “for much of its history most computer scientists saw it [neural networks] as vaguely disreputable, even mystical.” 4 It was recognized early in this history that the neural networks proposed during the 1950s and 1960s were limited by the comparatively minimal processing power available at the time. 5 The continued growth of computational power, along with the accumulation of large datasets during the 1990s and 2000s, played a major role in revitalizing progress in neural networks and motivating significant investment within the field of AI more broadly.
The field of computer vision, which focuses on advancing the ability for machines to extract understanding from images and video, offers one representative example on this point. “Traditional” approaches to these tasks in the 1990s and early 2000s focused on algorithms which specified
This content is AI-processed based on open access ArXiv data.
