Read Blog Post

Read Blog Post

Contributed on October 20, 2017

1.9K

681

Some thoughts on Emergence, Ants and the unintended consequences of AI and ML

Blog: Capgemini CTO Blog

Some thoughts on Emergence, Ants and the unintended consequences of AI and ML

Emergence models the formation of new patterns and macro behaviors, created bottom up from large numbers of small independent entities. The behavior of ants is a great example. I have been of the opinion that the biggest risk from AI and ML is the unintended consequences of large numbers of devices interacting in ways we couldn’t predict. Emergence models give us some view of how this might happen and perhaps nature also has some ways of preventing undesirable consequences.

Emergence, Ants and Cities

I listened to a radio program on Radio 4 last Sunday and heard a fascinating discussion on Emergence Models and how they can explain the way complex systems operate, when no one is directing or in control. Ants are a great example, every ant being an autonomous object with very low cognitive powers. They operate randomly testing things, searching for food and supporting procreation and survival of the colony. However, the way they operate is that once one ant bumps into something interesting it lays down a marker (ephemerons) and these markers draw other ants to the opportunity. This process repeats to good effect. This led to a discussion of the mathematically proven power of crowds and their ability to come up with a very accurate answer as a mean of all possibilities. Stephen Scott Johnson has developed this into a series of models and has a book published on the subject Emergence: The Connected Lives of Ants, Brains, Cities and Software. This book was published in 2001 so it’s worth now revisiting these ideas in the age of Artificial Intelligence (AI) and Machine Learning (ML).

The need for (and risk of) speed

Whilst the concepts and mathematics of AI and ML have been around for many years the big change over the last 10 years and indeed the last 2-3 years has been the exponential growth in processing power (driven by Moore’s Law) and in high speed network connectivity. This has led to the rapid growth in devices and systems capable of responding in near real time to events or information. This availability of devices, that one can interact with, is further fuelling the growth or systems which address specific needs, based on limited datasets and relatively straightforward ML algorithms. Bots is a good example, where systems with a limited use case are deployed to address specific customer services needs or resolve specific bottlenecks.

The downside is that we are already seeing unintended consequences as devices or programs interact faster than the control systems can manage. So called ‘Flash Crashes’ in markets due to mass algorithmic trading is a good example.
The world as a brain

So if these devices and systems are proliferating can we see something of scale beginning to take control. If we think of devices as individual neurons we can compare with the number that exist in different animals:

Estimates of the number of devices spread across the world vary but Gartner puts it at around 8Bn this year and other models put it as growing to about 75Bn by 2025. So on these calculations the world will have as many connected devices as a human being has Neurons within the next 10 years. Now Neurons in a human brain fire 5-50 times a second and most devices are currently not communicating at this sort of level. However, given the exponential growth of processing power and the speed of networks it wouldn’t seem unreasonable to expect the average device to start to operate within this range within the next 10 years.

So the Chinese celebration of animals for each year maybe prescient as the level of communications increases it is worth considering what will emerge.

A butterfly flaps its wings

So if we want to stop something emerging we didn’t intend what should we do about it? It is sometimes said that a butterfly flapping its wins in a Brazilian Rainforest can trigger an earthquake. However, in reality nature is a great example of a loosely coupled system, enabling different species to emerge in different parts of the world completely independent of other ‘worlds’. There is thus a level of isolation of each ecosystem to the other through the balance of time, space and rate of change and the damping effects of friction and other natural impediments to movement.

The danger we have is that all of our efforts in the internet world are aimed at removing impediments to communication of information and consequently we are actively removing any dampers. This headlong charge means that we are bringing forth the unintended consequences of connectivity at pace without any idea what will emerge.

So how does nature deal with things when they run out of control, such as swarms or overpopulation? There are maybe two extreme models to think of, either the individual entity autonomously deals with it itself (Lemmings) or one population wipes out another.

Oh dear!

So what does all this mean to us as individuals and as the human race. Perhaps we have a number of options to choose from if we want to avoid unintended consequences from the proliferation of AI/ML enabled IoT devices:

1. Assume it’s not going to happen
2. Recognise the possibility of emergence at a local or global level but ignore it and hope for the best, its someone else’ problem
3. Try and build natural dampers into the systems to segment the problem
4. Build some self-destruct features into devices to ensure self-regulation or fail safe.

Oh dear quite a difficult problem, one to mull on.

References
https://www.theguardian.com/science/blog/2012/feb/28/how-many-neurons-human-brain
https://en.wikipedia.org/wiki/List_of_animals_by_number_of_neurons
https://www.statista.com/statistics/471264/iot-number-of-connected-devices-worldwide/
https://www.gartner.com/newsroom/id/3598917
http://www.human-memory.net/brain_neurons.html