Modern artificial intelligence is built to mimic nature—the field’s main pursuit is replicating in a computer the same decision-making prowess that humankind creates biologically.
For the better part of three decades, most of AI’s brain-inspired development has surrounded “neural networks,” a term borrowed from neurobiology that describes machine thought as the movement of data through interconnected mathematical functions called neurons. But nature has other good ideas, too: Computer scientists are now revisiting an older field of study that suggests putting AI through evolutionary processes, like those that molded the human brain over millennia, could help us develop smarter, more efficient algorithms.
But first, back to middle-school biology class. The concept of evolution, famously credited to Charles Darwin and refined by countless scientists since, states that slight, random changes in an organism’s genetic makeup will give it either an advantage or disadvantage in the wild. If the organism’s mutation allows it to survive and reproduce, that mutation is then passed along. If it doesn’t, the mutation dies with its organism. In algorithm world, this is known as neuroevolution. While artificial neural networks replicate the process of learning individual concepts, neuroevolution tries to recreate the process that built parts of the brain—i.e. the process by which only the strong (or smart) survive.
Although neuroevolution has been around since the 1980s, the concept is getting renewed attention as researchers dig into the archives for different perspectives on machine learning. In the past month, Google Brain and non-profit organization OpenAI each published unreviewed papers on the subject, Google’s on the application of neuroevolution principles to image recognition and OpenAI’s on using “worker” algorithms to teach a master algorithm the best way to accomplish a task.
🤖
Bringing biological evolution into the already complex field of AI research can be confusing. So if it’s easier, think of algorithms as horses. Horses learn throughout their lifetime, but they are only evaluated on a few different metrics, like how fast they run. Accuracy in image recognition is easy to assess as a single number, as is the amount of time it takes a horse to run around a track. But what actually makes that horse run faster is incredibly complicated—a vast network of DNA that enables muscle growth, higher stamina, and even intellect. That complexity mirrors the underlying parameters of algorithms, or how an algorithm might be good (or bad) at image recognition. So if you get lost anywhere in the article, just take a deep breath and think “horses.” (This is also good life advice.)
For its research, the Google team generated 1,000 image-recognition algorithms that were trained using modern deep neural networks to recognize a specific set of images. Then 250 computers each chose two algorithms and tested their accuracy by making them identify an image. The algorithm with higher accuracy lived, while the one that performed poorly was “killed.” The survivor was then copied, and its clone (or “child”) was changed slightly—just like human DNA randomly changes during reproduction. But instead of blue eyes or a widow’s peak, this mutation slightly altered how the new algorithm interprets training data. The clone was then trained using the same data as its parent, and put back into the batch of 1,000 algorithms to start the process over again.
Google researchers found that neuroevolution could cultivate an algorithm with 94.6% accuracy, and recorded similar (though not identical) results during each of four repeats of the experiment. Mutations that improved the algorithm’s image-recognition skills were rewarded (i.e. those algorithms survived) while mutations that decreased performance were killed off. Just like in nature.
The differences between the five sessions also illustrated a consistent problem. Google researcher and paper co-author Esteban Real says the algorithms kept getting stuck halfway through the process, seemingly unsure whether to continue evolving or scrap the mutation and start over. Real says an analogy in nature might be the evolution of wings. “Half a wing might not help you very much,” he says, “but a full wing lets you fly.”
Google’s team is now working on getting the evolutionary models to explore different mutations more fully (to build whole wings). But that gets tricky. The team only wants the algorithm to mutate in a limited way, so it doesn’t end up with a whole bunch of extra code that isn’t useful. Explains Real: “The worst would be having many half-wings.”
🤖
By focusing mainly on image recognition, Google tested both neuroevolution’s capacity for tackling something the biological brain is great at, and its ability to solve a modern problem. OpenAI, on the other hand, used a more pure form of evolution to undertake a different task.
Rather than training thousands of algorithms to get better at one thing, the OpenAI team wanted to use “worker” algorithms to train a master algorithm to accomplish an unknown task, like playing a videogame or walking in a 3D simulator. This technique isn’t the primary way to teach machines how to make decisions, but a way to teach them how to learn from specific information more efficiently, explains co-author and OpenAI researcher Tim Salimans. The evolutionary algorithm is able to monitor how its workers are learning, and it essentially learns to learn—that is, to extract more knowledge from the same amount of data.
To conduct their research, the OpenAI team set 1,440 worker algorithms to the task of playing Atari. The workers played until they reached Game Over, at which point they reported their scores to the master. The algorithms that garnered the best scores were copied, as in the Google research, and the copies were randomly mutated. The mutated workers then went back into rotation and the process repeated itself, with advantageous mutations being rewarded and bad ones killed.
This approach also has its limitations, chief among them that the worker algorithms only report one number, their high score, back to the master algorithm. The algorithms with the best scores survived, but trying to make the master aware of any specifically successful moves would require far too much computing power. (In biology, the parallel might be an ant colony: Workers go out and find the most optimal solutions; the queen is the central hub of information.) In other words, OpenAI learned a lot about success, but less about scrappiness.
🤖
Back in the 1980s, neuroevolution and neural networks were similarly sized fields of study, says Kenneth Stanley, an associate professor at the University of Central California and a recent addition to Uber’s AI team (through their acquisition of Geometric Intelligence, which he co-founded).
“There was a small community of people who thought about how brains, which are really the only proof of concept of intelligence in nature, get into the world,” Stanley says. “Some people thought maybe the most straightforward way to do this would be to create a evolutionary, Darwinian-like process in a computer that acts on little artificial brains.”
Neural networks took off when three computer scientists—David Rumelhart, Geoffrey Hinton, and Ronald Williams—published a 1986 paper describing an algorithm that enhanced the way those networks learned from their mistakes, called backpropagation. The findings greatly improved the efficacy of hand-built neural nets, but an impending AI winter—funding was slashed for purported lack of progress—impeded further growth. It wasn’t until the mid-aughts that Hinton and company began publishing papers that made neural networks too enticing for the larger computer science community to resist, showing that backpropagation allows neural networks to grow immensely, and to in turn understand far more complex ideas. These networks were dubbed “deep,” and deep neural networks became the most popular flavor of modern artificial intelligence.
“Because of that, there was some loss of awareness for neuroevolution, which was this parallel thread of evolving brains,” says Stanley.
Back in 2002, at the start of his career, Stanley wrote an algorithm called NEAT, which allowed neural networks to evolve into larger and more complex versions over time. His corresponding paper has more than 1,600 citations on Google Scholar, and has been referenced in deep neural-network design and neuroevolution research since its publication. In 2006, Stanley published Hyper-NEAT, an algorithm that made neuroevolution much greater in scale, inspired by DNA’s ability to be the blueprint for billions of biological neurons with trillions of connections, despite only having around 30,000 genes. (Fun fact: Hyper-NEAT’s full name is Hybercube-based NeuroEvolution of Augmenting Topologies. I challenge anyone to name a better algorithm anagram.) Today, Stanley says it’s gratifying to see his career’s work back in the zeitgeist.
Like Stanley, OpenAI and Google are now tackling two different ideas plucked from the same field. Google’s hybrid approach combines classic neuroevolution with the techniques, like backpropagation, that have made deep learning so powerful today: Teach an algorithm how to act in the world, let it evolve, and that algorithm’s child will have most of the accrued knowledge. OpenAI’s approach was more true to how evolution works in biology. The team only let the randomized mutations in every generation govern how the networks improved or failed, meaning improvement was only created through random evolution. But both attempts had very clear goals—recognize an image, or get a high score in a game (or make a horse run faster). How the algorithms got there was up to nature.
“Individuals are born with the weights in their brain that they’re going to have for their entire life,” Stanley says about OpenAI’s work. “It’s like if we bred you, and your children, and your children’s children, and then they knew calculus.”