In any number of industries, from transportation to medicine to defense, advocates are calling on artificial intelligence to play increasingly large roles as the fields look to the future. But a new study shows there's a dirty side to AI: energy consumption.
The act of training a neural network, according to the study led by Emma Strubell of the University of Massachusetts Amherst, creates a carbon dioxide footprint of 284 tonnes—five times the lifetime emissions of an average car.
Neural networks are sometimes considered the "brute force" hammers of the AI world, effective but inefficient. Strubell's study examined four different AIs all involved in understanding written words: Transformer, ELMo, BERT, and GPT-2. GPT-2, for example, is trained to predict the next word in 40 gigabytes of Internet text. It was trained to process words from 8 million web pages.
Strubell's team measured each AI for one day and sampled the energy consumption throughout. That figure was then multiplied by the length of time it took to train the AIs, according to their original builders. From there, the carbon footprint was based on average carbon emissions used in American power production.
Scientists identified a process known as a Neural Architecture Search (NAS) as the main offender. Using trial and error, NAS automates the design of a neural network. Referred to as one of the "new kings of the deep learning castle," NAS is seen as a "quick and dirty way of getting great accuracy for your machine learning task without much work."
However, training the Transformer AI, which is focused on translating between languages like French and English, with NAS is vastly more time-consuming. It only takes 84 hours to train the Transformer without NAS; with the process, it can take over 270,000 hours.
NAS is fine-tuned for extremely specific tasks within the AI framework—translating German or French, for example—meaning that expends 3,000 times the amount of energy.
There are limits to Strubell's study, which will be presented at the Annual Meeting of the Association for Computer Linguistics in Florence, Italy in July. Giant companies like Google and Amazon present the option to researchers of paying for cloud-based platforms that can be used to remotely train AIs. Understanding how Google and Amazon use these cloud platforms would paint a fuller picture.