Serendipity: a bumpy ride

What I give form to in daylight is only one per cent of what I have seen in darkness.”

M. C. Escher

If you ask a scientist what is the weirdest experience in their research activities it is most likely the answer will be how good ideas sometimes come suddenly. This is certainly a strange experience, especially when compared to methodic and analitical work, that includes comparison and criticism, which tends to occupy the conscious rational mind. You possess a knowledge in your field, you know how to apply the techniques and you think you have a clear idea of what the question is. And then very often you find yourself caught somewhere in the middle, or perhaps very close to the solution you arelooking for. You decide to leave it for a while, you take a break, and apparently nothing happens … until suddenly, without thinking about it, you clearly see that youknow the solution. Something has woken up the brain that was apparently far away fromthe problem, and it seems to have continued working outside of yourself. It is triggered sometimes by a formula on a blackboard, or a banal comment from a friend, perhaps bythe shape of a stone on the road or reading a text that has nothing to do with the problem. Without noticing, and fired by a fortuitous event, the problem that fighted a solution or even seemed impossible to approach is no longer resisting and the solution is shown last, diaphanous and transparent.

Research often involves travelling a path that others have opened up to this end. From here, you must decide where to continue exploring. Research is therefore going to the extreme of knowledge (or ignorance) and in order to succeed we need to know how to successfully trace the path through this map of the unknown. But as it happened with the old maps filled with blanks, the way forward is full of uncertainties. The more difficult it is, the greater the white space. Passing through this terra incognita is inevitable, as it is inevitable to be mislead. The surprising things is that sometimes, at the end, we haven’t lost our way and after walking in the dark, our destiny appears before us just likean oasis in the middle of the desert. It is not surprising that this experience has triggeredmultiple interpretations among scientists and non-scientists.

Beside the daily way of scientific progress, based onprevious knowledge that operates just like adding bricks to the building, there is a different way of finding the answersthat seems to operate outside the logic of conscious thought. This perception of a parallel process that escapes conscious perception is reinforced with some well-known examples. One is the one of chemist Kekulé, that sought the chemical formula of benzene, a compound of which its composition was known, but not its structure. He proved the later to be a closed ring. Kekulé said he had reached the correct formula (by mid nineteenth century) after having a dream in which he saw a snake biting its tail. This serendipity resulting from a dream has been quoted many times and adds to the mystery of the phenomenon.

Serendipity, or more specifically, the concept of innovation has been the object of scientific inquiry itself over the last decade. The underlying problem is much more general and seems to be involved in various phenomena that go from artistic creation to the evolution of viruses, and is the nature and origin of the sudden jumps that introduce deep innovations. So, although the evolution-as proposed by Darwin- is held gradually in most cases, it also has transitions that indicate that the system has “discovered” something special. Evolution operates unintentionally (it is “blind”) so these abrupt transitionswhere a new design or a response to a changing environment are invented are both random events. But it is impossible not to see the connection with serendipity associated to human creativity. Can we explain this phenomenon? Can we ever take advantage of this knowledge to improve the chances of a discovery? In this sense significant efforts have been undertaken, with the aim to determine the structure of the space of possibilities where viruses and humans beings move in search of new solutions.

This approach seems to be very promising is the study of the space of possibilities in terms of so-called “adaptive landscapes”. In abstract, we can imagine these surfaces as the total space of possible solutions to our problem and we observe the peaks that provide a measure of how these are optimal solutions. During the research process, we can imagine we are walking on its surface and always trying to reach a close peak.

We only change for the better. For us it is only visible what it is closer, so what generally happens is that we almost always get caught in a peak of little height (as the best and highest are also rare). Thinking this way helps us to understand some of the peculiarities of the search for solutions and maybe of creativity itself. As it has been pointed out by physicist Murray Gell-Mann, when exploring a space of possible inventions or ideas, the mind wanders randomly (though it is certainly led by our knowledge of the problem) until it finds a solution. The problem is that generally, when we reach a peak we are likely to get caught in it instead of improving. If the solution does not satisfy we might stay blocked and probably abandon at some point. This is where unconscious brain activity plays a key role At this level, what might happen is that the research method relaxes enough the rules in order to let the mind escape an unwanted peak. We are now moving along a less direct path, probably passing along small peaks.

In the field of complex systems, we say that we generate “noise”, which equates in space research to unarranged jumps. The loss of stiffness and a bit of luck can mean that, by chance, we get to one of the major peaks. We have had a great idea!

When physicists of complex systems have studied the structure of these landscapes they have discovered that these landscapes have a structure that allows for the understanding of the process of sudden change. What has been observed in these spaces is that large areas of little changes in relief (neutral) coexist with other similar ones that can be accessed through very special transitions. Some researchers, inspired by general relativity, have called these thin tunnels that connect two major neutral areas “wormholes”. Generally, we will move through space within a neutral (or network) zone, with few changes. But if we get to a wormhole (serendipity!), we can jump to a new neutral zone which, hopefully, will be much better. This type of behaviour is seen in many real systems, including viruses and technological systems. It may also be responsible for the curious idiosyncrasies of our way of thinking. And this observation of transitions provides a possible explanation to the proposal of palaeontologists Stephen Jay Gould and Niles Eldredge that they have called “theory of punctuated equilibrium” in which the slow and gradual change would be replaced by great leaps that happen on a small scale.

Are there recipes for improving serendipity? If the previous metaphor of landscape is valid, then there are some basic guidelines. On one hand, it is advisable to have knowledgeable partners to discuss the problem and to tackle it using different views. This step will allow us to expand our area of research. Likewise, the risk of being caught in a peak, perhaps because what we get fits into the idea of our knowledge of the problem and expectations (which can be counterproductive) can be reduced by simply trying to dismantle our own idea or trying to adopt the remotest possible view from our own.

Beyond these basic rules (which are not always used) what seems to be clear is that it is positive to let our minds wander freely. So alternating periods of deep analysis and revision of ideas with phases of neglect will help the task. In the future, we will expect new advances in the scientific study of innovation and possibly also to see what new technologies can contribute.

This is the case, for example, of future artificial intelligence systems that may serve –perhaps in a different way that our mind does- as an alternative source of finding solutions. What we can extract from these artificial systems can be beneficial for us and probably bring unexpected results. As shown by an experiment in the evolution of robots competing for energy resources in a given space, some robots discovered by chance a way to improve its performance: by lying. A notable serendipity finding for an artificial system that, going beyond what any of the scientists who designed them had imagined, discovered one of the things that one way or another makes us human.


J. Crutchfield. 2000. “When Evolution is revolution”,

Evolutionary Dynamics. Oxford University Press, New York.

M. Gell-Mann, 1995. The Quark and the Jaguar. St. Martin’s Griffin, New York.

S. Kauffman. 1995. At home in the universe. Oxford University Press, New York.

S. Kauffman. 2003. Investigations. Oxford University Press, New York.

R. V. Sole & Brian Goodwin. 2001. Signs of Life. Basic Books, New York.

All rights of this article reserved by the author

View comments0

Leave a comment