Complex systems and creativity.

In: Science, Mind, and Creativity: The Bari Symposium ISBN 978-1-60876-115-9 Editor: L'Abate, De Giacomo, Capitelli and Longo © 2009 Nova Science Publishers, Inc. Chapter 3 COMPLEX SYSTEMS AND CREATIVITY Savino Longo, PhD.* Professor, Department of Chemistry, University of Bari, Via Orabona, 4; 70126 Bari, Italy In a symposium about science, mind and creativity it is appropriate to include some ideas on the topic of complex systems. In this chapter the author will discuss a few aspects of complex systems and describe in more details those developments, which have a more direct connection with the themes of the symposium. The main argument in the chapter is that the creative process has its very root in the natural laws, especially under conditions of limited resources, which impose a conflict between alternative organization patterns in systems. After a general introduction, the author will illustrate the methods and the results of many computer-based experiments, starting from physical and chemical systems, moving on to models of brain functions, the evolution in a simulated community of protozoa and the use of ideas from genetics to solve complex problems. STRUCTURES An important problem in natural sciences is the development of structures in the World, which looks to contradict the thermodynamic predictions of a progressive increase of disorder (Nicolis & Prigogine, 1989). The second law of thermodynamics predicts in fact that all deviations from homogeneity in systems where the mixing is not constrained by boundaries will eventually fade away: an example is the mixing of ink with water after a drop of the former is poured into the latter (Figure 3.1), or the homogenization of temperature between a hot and a cold body into thermal contact. * Tel: +390805443563; Fax: +390805442024 Email: [email protected]  2 Savino Longo Drawing by the author. Figure 3.1. Mixing. All these processes are mathematically described by the increase of a function called entropy; therefore the second law can be stated as the law of the increase of entropy (Dickerson, Gray & Haight, 1973). Since the second law of thermodynamics never failed in predicting the outcomes of laboratory experiments, the presence of structures, patterns and shapes in nature needs an explanation. The previous statement can easily be misunderstood, since limited differentiation in some systems is not excluded by the second law, but in fact it is predicted on the basis of the increase of entropy: for example, setting in contact two non miscible fluids like water and oil leads to two separate phases, each with a different composition. Indeed, this limited kind of separation can even emerge from uniformity, as a consequence of the increase of entropy. This happens when mixing two solutions, which produce an insoluble substance (e.g. a solid precipitate, like in many analytical reactions used to detect specific chemicals present in solution) or when evaporating salted water until salt crystals appear on the bottom. Therefore the problem is usually stated as that of the formation of complex structures, although for a more exact and unambiguous formulation some mathematical tools would be necessary but are not appropriate here1. More loosely, a complex structure can be defined as something which presents many shape discontinuities or variations in some way unpredictable but not genuinely random. The examination of several examples in the following will help to clarify this point and there will be little ambiguity at the end, is spite of the lack of a formal definition. Even better, it will become clear that the complexity of a structure is to be judged not merely based on its shape, but it is better based on the evaluation of exchanges of energy or information. The creation of ordered structures has been explained in two distinctly different ways for very large structures in the universe and for smaller structures, from astrophysical flow patterns to living structures. Very large structures, like galaxies, which are formed under the action of gravity, are explained by invoking a peculiar physical property of self-gravitating systems, i.e. their negative thermal capacity (Fang, Xian & Kiang, 1989). This technical term  Complex Systems and Creativity 3 means that the average velocity of bodies in a self-gravitating system increases when energy is subtracted from the system, while the opposite holds up generally. The negative thermal capacity involves an inversion of the thermodynamic predictions: for these systems the entropy increase is associated with the formation of structures, rather than with their decay. The mechanism described above applies to self-gravitating system, i.e. systems where any particle is interacting with all the others. A different mechanism of structure formation applies to systems of particles, which are attracted by a limited number of bodies but not with one another. Structures can develop in this case by the mechanism of resonance (Gutzwiller, 1991). When a periodic force is perturbing an oscillator, the response of this last is much more intense if the ratio of the frequencies of the oscillator and of the force can be expressed by a rational number, i.e. a fraction of the form n/k where n and k are both integer, with small values of both n and k. The set of this numbers into the bigger set of real numbers (any number in decimal form, e.g. 1.543…) has a beautiful complex structure, and a more or less deformed image of such structure is what is observed in the dynamics2. The phenomenon applies also when the perturbation is not periodic but a kind of pulse, since it can be shown that a pulse can be expressed as a combination of periodic components. For example, Figure 3.2 shows a single frame obtained from a computer simulation, where an initially uniform, flat disk composed of small particles orbiting a compact object (in a way similar to Saturn and its rings), is perturbed by a second, passing-by compact object. This last moves perpendicular to the initial plane of the disk; at the time represented in the frame is leaving upward and it is no more visible. In the simulation any particle is represented by a mathematical point with its position, velocity and mass, which is moved by small time steps, taking into account the forces acting on it, namely the gravitational attraction towards the compact object and the passing-by, perturbing object. What is seen in the picture is a two- dimensional, perspective rendering of a three-dimensional complex structure. By running the simulation it is possible to view the tree-dimensional differentiation of the initially uniform disk as it proceeds in space and time. A similar approach has been used in the past to study the structures obtained from the collision of two galaxies: in this case dots represent stars and the two compact objects are the mass centers of the two colliding galaxies (Toomre & Toomre, 1973). It is obvious that the chance to observe such beautiful complex phenomena is a strong incentive to make the effort to develop the computer simulation code. This is not a very simple task since it requires some skill with a computer programming language and a long series of attempts to select the appropriate representation of the phenomenon. We can say that there are two creative processes involved in the realization of a visible artifact like that in Figure 3.2: the creative role of the dynamic laws of gravitating systems and that of the researcher striving to get a significant and aesthetically pleasing sample image of such dynamics.  4 Savino Longo Computer simulation by the author. Figure 3.2. Example of gravitational complex structure. These gravitational mechanisms, though fascinating, cannot explain the development of smaller structures where gravity can sometimes play a role but it is never the only essential factor. Such structures can be explained, instead, by recognizing that the local conditions where such structures develop are not those described by the laws of equilibrium thermodynamics but are characterized by strong, sustained non-equilibrium conditions. For example in the biosphere, the radiation produced by the Sun (about 6000°C) coexists with the lower temperature of the Earth environment (average 15°C) and with the much lower temperature of the night sky (-270°C). In strict thermodynamic terms, the entropy production associated with the huge heat flows present under such conditions is sufficient to balance all the morphogenetic processes in the Sun system, avoiding any contradiction with thermodynamics. This point is tricky and maybe needs to be better clarified: structure creation is in local contradiction with thermodynamics, since at the local scale a structure is formed or sustained in a medium, but this process occurs only if a flow of heat or a process physically equivalent to a flow of heat occurs through the medium itself. This means that an amount of entropy is being produced, since the flowing of thermal energy from a warmer body to a colder one is always associated with the increase of entropy, and the entropy increase per unit time associated to the global process is always positive. If the flow of heat is interrupted the structure disappears, since the only process now active, the local one, is still subjected to the second law in its evolution in isolation. Therefore no problem is arising at this scale of length and time as regards thermodynamics: at the same time, the local development of structures, ranging from  Complex Systems and Creativity 5 planetary wind circulation to Life, is of the highest interest and provides a link between the fully coherent yet somewhat arid thermodynamic scenario in which structures develop and the endless variety of meanings which can be associated to complex structures and their interactions. An example of physical structure development, which is often considered as a pedagogical start, is fluid convection. Morphogenesis in fluids is a striking example since fluids are also the media where structure fading as predicted by the second law is fastest, because turbulence in liquids and molecular diffusion in gases allow a fast mixing of different miscible components into each other (like ink and water considered before). Let us consider a glass vessel containing some water, which is heated from below, for example by a gas burner (see Figure 3.3): as stated above for the case of the Sun system, the sustained temperature difference between a heat source (here the burner flame) and the colder environment is essential to produce an effect. After some time, a flow pattern is established in the form of convection “rolls” where hot water coming from the bottom of the vessel rises without hindering the descend of cooler water coming from the surface where the thermal energy is dissipated into the environment (Haken, 1983-a, Haken, 1983-b) (Figure 3.3). Drawing by the author. Figure 3.3. Convection “rolls.” Similar structures on a much larger scale are observed in the Earth atmosphere and are taken into account by glider pilots and birds. These structures are cognates to the vortexes  6 Savino Longo formed in viscous fluids under a shear stress, which are observed after the passage of a ship on the surface of water3. These vortexes are also observed, on an impressively larger scale, in the beautiful probe photographs of the atmospheres of the big, gaseous planets in the sun systems, in particular Jupiter and Saturn. In the upper, visible layer of the atmosphere of Jupiter the circulation patterns are well evident even in color photograph (Figure 3.4) since, depending on the local atmospheric temperature and winds, the gas mixture (mostly hydrogen) is richer of several kind of colored (like phosphorus and sulphur) and white (mostly solid ammonia) impurities (Strycker, Chanover, Sussman, & Simon-Miller, 2006). Figure 3.4. Atmospheric circulation patterns on Jupiter (Courtesy JPL/NASA). From the cold ad huge-scale realm of planetary atmospheric circulation we now shift to a different example, which is more closely related to the pattern development in the realm of Life. A few decades ago much attention was attracted by chemical reactions producing a periodic color change of a solution, important instances of which are the Belousov- Zhabotinsky (BZ) reaction and the iodate reaction (Epstein, Hustin, DeKepper & Orbán, 1983). When the mixture is well stirred, these reactions behave like “chemical clocks” producing structures in time rather than in space (Figure 3.5). Note that in this case no thermal source is present, but the entropy production is due to the chemical reaction: this is not a surprise for those who are acquainted with some basic chemistry and will recall that a fixed amount of specific entropy is associated to any chemical substance, and that the total entropy increase is the force driving the chemical reaction to completion.  Complex Systems and Creativity 7 Drawing by the author. Figure 3.5. Chemical oscillations. As in the previous example, however, the special reaction conditions associate this irreversible global entropy increase with the transient, or local, appearing of ordered structures. When the reaction is allowed to occur in a shallow vessel and no stirring is provided, these reactions produce fascinating structures evolving in space and not only in time: these structures have typically the appearance of moving strips, changing spots and rotating spirals, depending on the feeding of reactants and the depth of the solution in the vessel (Epstein et al., 1983). A simple theory for these chemical structures can be provided by modeling the reaction vessel as an array of cells, and assuming that diffusion is bringing chemicals from any cell to the two side ones, while oscillation is going on. The description of the process can be reduced to minimal terms by just specifying rules for coloring the cells in a row of a matrix of cells based on the pattern found in the row above representing the immediately preceding state of the vessel (i.e. the horizontal reaction corresponds to space, the vertical one to time, increasing downwards). These models, called cellular automata, are known to produce the most various and surprising structures (Schroeder, 1992). The reader can be interested to try a simple example. Let us start from a sheet of white paper with two arrays of intersecting horizontal and vertical ink lines individuating squared or rectangular white cells. The rules are: a cell in a horizontal row is filled black if and only if a single one of the two diagonally placed above is black. These rules look reasonable for a minimal description of the system under  8 Savino Longo consideration: a colored chemical diffuses sideways while its concentration increases due to the reaction (case 1, upper left side of Figure 3.6) but if the concentration rises too much the colorless situation is locally reestablished (case 2, upper right side of Figure 3.6). Now, start from a single black cell in the middle of the uppermost row. After some work, a pattern like that shown in Figure 3.6 appears. Figure 3.6. The structure produced by a coloring procedure inspired by chemical kinetics, see the text for details. This geometrical structure is called a Sierpiński gasket, and it is an example of fractal (Schroeder, 1992). A fractal is a structure with a non-integer topological dimension D. The topological dimension is the extension of a well known elementary concept, the integer dimension of a geometric object, based on the minimal number of coordinates necessary to find a specific place on it, and equal to one for a line, two for a flat surface, three for a volume-filling shape. While an exact calculation of D can be made by using the appropriate mathematical theory, it is evident from the Figure 3.6 above that the Sierpiński gasket is somewhat “emptier” than a compact figure in the plane yet “fuller” than a twisted line: therefore we expect4 1<D<2. Similarly, it can be demonstrated that the chemical aggregate, which will be considered in the next section, has also intermediate dimension between 1 and 2 and it is a fractal. Non-equilibrium structures associated to chemical processes have been invoked as the causes of skin color patterns in animals like zebras and leopards and surface color patterns in some kinds of shells (Camazine et al., 2003). Indeed the resemblance of the Sierpiński gasket with some shell color patterns is striking.  Complex Systems and Creativity 9 CONFLICTS An interesting aspect of these phenomena is that the scheme according to which matter is organized is a dynamic one. There is little connection between these non-equilibrium structures and static, equilibrium structures like crystals. The crystal structure, in fact, is a necessary one, which is strongly connected to the chemical nature of the matter sample under consideration. So, for example, sodium chloride (NaCl) crystals have a cubic geometry, which is imposed by the ionic radii of sodium and chloride ions (Dickerson et al., 1973). On the contrary, non-equilibrium structures are conditioned by general properties of the substance under consideration, like density and viscosity, but are often determined by the history of the system (i.e. the process by which the system reaches the present conditions). A critical issue is therefore the selection of the scheme of organization emerging under specified conditions, amongst a wide set of possibilities compatible with the boundary conditions (the mathematical specification of matter dynamics on the walls limiting the accessible volume). This selection can be expressed in terms of competitions between schemes, connecting complex systems with Darwin’s evolution. This connection has been considered and discussed by I. Prigogine (1997, Prigogine & Stengers, 1986) and H. Haken (1983-a, 1983-b). Both scientists attempted to find a general principle for non-equilibrium morphogenesis, but the two theories they formulated are very different. Prigogine, winner of the Nobel prize in chemistry for his studies on thermodynamics, attempted to formulate a general principle for structure development based on a thermodynamic quantity, namely the entropy production. He regarded these structures as the archetypal basis of the structures of Life, and formulated the idea that they emerge thanks to their ability to dissipate energy; for this reason he coined for them the term “dissipative structures”. Haken, instead, was inspired by the concept of adiabatic elimination in physics: a physicist can get a general view of the dynamics of a system concentrating on the variables which have the slowest evolution, since all others can only reach, at any time, a steady condition compatible with the present values of the slow relaxing ones. Accordingly, he supposed that a general principle for structure formation was to be found in the preliminary emerging of slow patterns of activity, which he called “order parameters”. This last mathematical concept and the related terminology were borrowed from the Landau’s theory of phase transitions. The result was no less than a proposal for a new scientific discipline, which he called “Synergetics”, whose aim was to study the emergence of collective behavior in systems of any kind, from fluids to human societies. Prigogine and Haken shared the idea that a general theory of structure development would have contributed to join hard and soft sciences, but they had complementary views of the way this process would have occurred. For Prigogine, the most important step was the introduction of a historical element in the description of natural systems. Accordingly, the methodology of soft sciences could have given its contribution in the understanding of chemical and physical systems. For Haken, the crucial step was to change the language and methodology of soft sciences, which would have adopted those of Synergetics.  10 Savino Longo Either ways, most contemporary scientists believe that the ambitions of both theories are not vindicated. Nevertheless, some common aspects of the methodology of both theories have strongly influenced further developments. In both these author’s views, apart the important differences seen above, the general idea is that the possible structures are the result of the nonlinear evolution of linear “modes” which describe mathematically the fluctuations of the equilibrium system, a generalization of Fourier analysis of a generic function, where all possible shapes of a vibrating system are decomposed in simple vibrations, or harmonics (Figure 3.7) Drawing by the author. Figure 3.7. Oscillation modes. The modes, which enter into a competition to determine the dynamic structure of the system, are those compatible with the boundary conditions, or loosely speaking with the geometry of the system, nearly in the same way as the harmonics of a vibrating string are compatible with the fact the string is fixed at two extremes. The modes compete to reach the most effective coupling with the external environment. The measure of the efficiency of the coupling can be defined in different ways, for example, as already stated above, in terms of the entropy production rate (Swenson, 1992) but for any specific model of the system it is always specified. The competition arises from the non-linearity of the laws governing the system. Due to the non-linearity one particular mode can emerge and dominate the subsequent dynamics of the system and the energy, matter or information exchanges with the external world. The emerging mode can have quite a complex structure but often it is recognizable as a deformation of a Fourier mode of the perturbations compatible with the boundary conditions. So for example the pattern of motion emerging from fluid convection is similar to an oscillation mode of the fluid, and the same can be said of the colored strips, which can be seen from the above in a shallow fluid where an oscillating reaction occurs.  Complex Systems and Creativity 11 Drawing by the author. Figure 3.8. Emerging modes. Left: convection, Right: chemical strips. This is schematically shown in Figure 3.8, where the horizontal axis in both Cartesian plots is the position, while the vertical axis is the vertical fluid speed in the fluid convection case and the color intensity for the chemical strip experiment. In other cases, like organization patterns in analog models of the brain, it can be hard to describe the emerging pattern as an evolved mode. But it is always important and inspiring to remember the role of mode competition in the process of self-organization. The creative role of competition for resources in the creation of complex structures becomes very clear in the process known in physical chemistry as diffusion limited aggregation (DLA), a model for the accretion of the solid phase from a solution or a melt, in some conditions of crystallization (Dewdney, 1990). The DLA is a growth regime where “germs” of solid phase form in advance in the liquid phase and diffuse until they meet the surface of the growing solid, where they stay. Another case where the mechanism applies is the growing of a solid metal from a solution of its ions, e.g. Ag from an Ag+ solution. A model for DLA is based on the representation of the diffusion process of the germ in the solution based on an analog game, like a piece in a game of checkers. The movement of the germ is completely random, and can be considered as the result of a throw of a single dice, where the outcomes 1, 2, 3, 4 are associated to the directions north, east, south, west respectively, while the outcomes 5 and 6 produce no effect (Figure 3.9). This dice is actually simulated in the computer model by using a mathematical formula, which produces random numbers, statistically indistinguishable from the outcome of a throw of a real dice. The possibility of generating such random numbers as a part of a mathematical simulation was perfected and exploited by the mathematicians John von Neumann and Stan Ulam, making possible a full digital computer simulation involving random elements, the celebrated Monte Carlo method (Eckhard, 1987).  12 Savino Longo Drawing by the author. Figure 3.9. Germ movement in a DLA model. The simulation of the accretion of the solid phase is realized by stopping the diffusing germ when it meets another particle, which is part of the solid: from this moment on the germ is now itself part of the solid and will not move anymore. Another germ enters the simulation in the liquid side far from the solid and the simulation proceeds. In Figure 3.10 it is shown a frame of the graphical output obtained with a computer program realized by the author, based on the rules just explained. In the figure the black background is the liquid phase while the colored shape is the solid. The crystal germs (not shown) enter the simulation domain from random positions along the upper side of the picture. In the initial condition only the bottom side of the domain is filled with already crystallized solid (bottom horizontal line). The color pattern in the picture is based on assigning a different color shade to any cell in the solid phase depending on the time at which the cell became part of this phase. The result of the computer simulation is a structure of remarkable complexity in view of the simplicity of the rules governing the model accretion. The complexity of this structure can be explained in terms of competition for resources. An accretion with an involved shape, like intricate solid filaments extending in the liquid phase, is able to intercept much efficiently the moving germs in the liquid, much better than a more compact shape. Since the growth of the solid structures is based on the absorption of the diffusing germs, the more complex shape is the most suitable for using resources in the environment.  Complex Systems and Creativity 13 Computer simulation by the author. Figure 3.10. Structures resulting from a DLA virtual experiment. MINDS In the selection process of the structures considered previously the relation between the final shapes and the rules determining them is still relatively transparent: convection rolls in fluids develop under the competing pushes of ascending hot fluid and descending cold fluid, strongly ramified aggregates are selected as optimal collector of diffusing particles. A less transparent, yet extremely interesting, kind of structure appears in historically relevant, simple models for some specific brain functions developed in the late 50’s. The most well known example is the perceptron, a device invented in 1957 at the Cornell Aeronautical Laboratory by Frank Rosenblatt (Minsky & Papert, 1969, Khanna, 1990). The basic idea was to develop a machine that can learn, like people, based on the presentation of examples which needed to be classified according to simple yes/no questions: for example “Is this item blue? Is this item an apple?”. In an attempt to obtain an analogical model with some biological plausibility, and not only a model for the function of learning, Rosenblatt proposed an architecture which was based on a network of simple machines, emulating neurons, which were cascade-connected and organized in layers of activity. This architecture (now known as a “neural network”), was suggested to him by the known histological structure of the neuronal system for the first elaboration of visual  14 Savino Longo information in the retina, and functionally by the Hebb model of the plastic connection between neurons (Hebb, 1949). In the original perceptron scheme a layer of cameras, each collecting light from a small part of the presented image, is connected to a first layer of neurons by a complex system of hard-wired, random but invariable electrical connections (Figure 3.11). A neuron is a simple machine (an amplifier, essentially), which sums up the signals it receives and produces a single output. Each neuron of the first layer is connected to a final neuron which is in charge of deciding if the presented figure falls within a given category: the answer is considered a “yes” if the final output signal is higher than a pre- specified discriminating level. The connections between intermediate and final neurons are of variable strength, obtained by interposing a variable resistor in serial connection. A perceptron learns the appropriate way to perform the required task by the presentation of many practical cases: after any case presentation those connections between units which contribute to produce a correct answer are reinforced, those who contribute to a wrong answer are weakened. After the teaching process an effective connection network emerges spontaneously in the machine, unlike other devices, which are able to perform the function using a solution “recipe” which needs to be developed in advance by a human specialist. The perceptron is quite limited as a practical devise for pattern recognition and classification, being able to solve only some tasks, those of the so-called linear separable kind. In view of its limitations, it was strongly criticized by Minsky and Papert in their influential book “Perceptrons” (Minsky & Papert, 1969), causing a momentary stop in the research on this topic, despite the fact that the criticism in the book was indeed quite constructive and, more remarkably, the fact that some of the limitations of perceptrons make them actually closer to the human experience of immediate vision. For example, perceptrons are unable to determine if an ink spot is outside or inside a labyrinth defined by a complex closed line, like most of us. Drawing by the author. Figure 3.11. A perceptron.  Complex Systems and Creativity 15 In spite of its simplicity, the perceptron is the forerunner of the contemporary neural networks. It was found in fact that most limitations of the original design can be overcome by making also the first layer of connection adaptable instead of hardwired. This makes a huge difference: in mathematical terms, the perceptron is essentially a linear machine, since the final neuron is just summing up the signals coming from the intermediate neurons, giving to each signal an appropriate importance which is determined from the learning process. When the first layer of neurons receives its input via adaptable connections, the neurons of these layer must purposely be made nonlinear (an amplifier with distortions) otherwise (it can be easily proved by some algebra) two layers of neurons are not better than a single one. Under such nonlinear conditions, when the full network reaches an optimal performance, it is usually observed a posteriori that several neurons of the intermediate layer specialized in the detection of “figure features” (Figure 3.12) Drawing by the author. Figure 3.12. Emerging specialization of a unit. A trivial example is the pre-classification of an apple as a roughly round object. What is very important is that the features used by the developed network to discriminate between objects are often non trivial and suggestive, and refer to global characteristics of a class of objects instead of concentrating on details. This element of creativity, intrinsic in a neural network, is due to the competitions of patterns of connections between neurons, and it is similar to the complex pattern development in physical and chemical systems considered in the previous paragraphs. These devices, either build up with electronic components or (most often) computer- simulated, are routinely used to perform many human-like tasks like the reading of handwritten texts, the recognition of defected samples on production lines, the coordination of movements of robotic arm, the driving of unmanned small vehicles, the matching of fingerprints or faces with a prototype (Floreano, 2001). A very suggestive example of the possibilities of neural networks is given by V. Braitenberg (1984) in his book “Vehicles: Experiments in Synthetic Psychology”. In this book the author describes “vehicles” of increasing complexity, living in an artificial environment, each with an “attitude” towards different opportunities and risks based on an  16 Savino Longo internal electric connection pattern. For example, a direct electric connection between two cameras placed on the front of the vehicle (its “eyes”) and two electrical engines used by the vehicle to move can produce a vehicle which “escapes” light sources or “acts aggressively” towards them, depending on the connection topology. But this is only the beginning: thanks to an imaginary material (the “ergotrix”) which allows to realize the adaptable connections requested by the Hebb model, these vehicles learn how to improve their performance and evolve to the point of developing a complex psychology, including “emotions” and “expectations”. A short review of ideas from Braitenberg’s book can be found in an article written by Dewdney (1987) and reprinted in one of his books (Dewdney, 1990). Some of the more recent applications are very far from the original aim of the perceptron as a model for the animal vision or learning. The present author used a neural network to “guess” the missing information in a sparse collection of chemical data (Armenise et al., 1992). In his 1995 book “The Engine of Reason, The Seat of the Soul”, the philosopher Paul Churchland speculated on the possibilities offered to social sciences by the use of neural networks to extract information from masses of data. Churchland shares with Braitenberg and with many other researchers (Edelman & Tononi, 2001), but not with all (Searle, 1994), the conviction that consciousness is the dynamics of a specialized neural network. EVOLUTION In the previous sections the interpretation of the spontaneous pattern development in terms of mode competition has been exploited, showing that structure development in complex systems has several points in common with evolution. Nevertheless, the examples above did not match the structure of evolution in a very important point: no role is played by the genetic memory and the population dynamics. Much more explicit examples of computer simulated evolution have been given in the past, and here the author will devote some space to a classical example. This example is also important in which it illustrates the spirit of creative thinking associated with naïve computer code development, and represents a paradigmatic sample of the kind of popular computer science which had a strong impact on a generation of scientists. Originally formulated by Michael Palmiter, this example has been popularized with the name “Palmiter’s protozoa” by the computer scientists and science writer A. K. Dewdney in his column on the journal “Scientific American” (Dewdney, 1989) and later reprinted in his 1990 book “The Magic Machine”. It is all about the computer-simulated life of simple creatures, named “protozoa”. Palmiter’s protozoa live, move and feed themselves in a simulated, two-dimensional environment. This last is structured like a game board similar to the one considered above when discussing the diffusion limited aggregation. The creatures are just pawn moving on this simulated board. In a single move they can shift to any one of six positions close to the starting one and placed on the vertexes of an hexagon (Figure 3.13).  Complex Systems and Creativity 17 Drawing by the author. Figure 3.13. Allowed moves of Palmiter’s protozoa. Any creature has an energy supply, which is spent for moving, and must not drop below zero, otherwise the creature dies, i.e. it is removed from the simulation. The creature can increase its energy supply by eating “bacteria”. Bacteria are immobile dots, which appear randomly on the board. Any creature, and this makes the real difference with respect to previous examples, has its own “genotype”: a sequence of numbers which represent the propensity of the simulated creature to proceed, on the direction towards which it is faced at a given time or rather turn clockwise or counterclockwise. This genotype, to gain biological plausibility, is never changed in the course of the life of the creature. Any creature is given several chances to reproduce itself, by simple scission, at an time of its life, provided it is in good heath, i.e. the energy supply is high enough. The scission produces another creature with a genotype very close, but not equal to, the one of the parent one. These simple rules suffice to generate on the screen of the computer an impressive dance of life and change, where the more apt specie emerges depending on the rule which is used to disseminate the bacteria on the board. This emergency is made more explicit by using a different color to picture any creature on the output screen, depending on its genotype. It is impossible to convey the impressions, which arise in the spectator’s mind, when observing this abstractly rendered yet very explicit, changing pattern of competition and change. But it is worth trying. Figure 3.14 shows two frames extracted from the output of a computer program based on Palmiter’s rules and written in Delphi5 by Pieter Spronck, of the Tilburg University centre for Creative Computing (TiCC) (http://ticc.uvt.nl/~pspronck). In the pictures the round objects are protozoa, while the smaller yellow squares are bacteria.  18 Savino Longo Figure 3.14. Palmiter’s protozoa. See the text for details.  Complex Systems and Creativity 19 The pictures give a very faint idea of this simple but amazing simulated world. The first picture represents the diffuse population of members of the essentially straight-line moving species (conventionally blue), which emerges when the food is randomly dispersed in the environment. This species is the fittest under such conditions, since by moving essentially straight its members collect more bacteria than those of other species, which are doomed to starve. By changing the rule for the dispersion of bacteria an impressive change is produced. In the second picture bacteria are much more concentrated in a small region of the board, the Garden of Eden, and very scarce elsewhere. In this new situation, the members of the straight moving species are no more efficient collectors, since they move most of the time in desert regions of the board. A new species emerges, whose members spend their life swirling in endless turns. These mutants are now much more effective than the previous species, since they colonize the Garden and collect the concentrated bacteria available there reducing the food to a level which is not high enough to feed the straight moving creatures, during their occasional visits in the Garden. In the meantime, a very few bolder individuals, or less specialized mutants, explore the external world were the food, though scarce, is slowly increasing after the extinction of the previous specie: they are most probably doomed to die of starvation, but in due time some of them could discover the new possibilities offered to a limited population by this wide, arid territory. A cyclic regime of extinction and mutation can be established under some conditions (depending on the ratio between the food distribution inside and outside the Garden), somewhat similar to the chemical clock of Figure 3.5. Palmiter’s protozoa were very popular 20 years ago. While they cannot match the presently available, remarkably complex simulated worlds, where much more powerful evolution rules are coupled to realistic models of the environment and impressive computer graphics, they are still a very effective way of illustrating the ideas of artificial evolution. CREATIVITY Although the previous example was surely suggestive, it represented only a very simple attempt to simulate evolution. As mentioned above modern simulations are much more complex, and even computer games have been produced, which allow to play with evolving creatures in a simulated world. Nevertheless, the scope of these simulations is still limited by the fact that the solutions found by the organisms are useful, or meaningful, only in their virtual world. The next step, a very bold one, would be to use evolution in a simulated world to solve real problems. Amazingly, this was proven to be possible. Computer simulated complex systems are already in use to produce new ideas for real- world technological problems. This further step was facilitated by the following circumstance: many difficult problems in modern technology have a combinatorial nature, i.e. the problem consists in exploring a huge space of possibilities in order to find a scheme which best fits some well known requirements. This is the case of electronic circuits, like those used to generate signals of appropriate shape or to give an appropriate response to such signals (Koza, Bennett, Andre, Keane &  20 Savino Longo Dunlap, 1997). These circuits are made up of components, namely resistors, capacitors, transistors, each with a specific function. Any component is provided with two or more electric terminals, depending on its type. A circuit is made up by connecting the terminals of an ensemble of such components, in a way suitable to fulfill the requested function. The requirements to be met by a circuit proposal are of several different kinds but they can all be expressed as functions of the element number and disposition in the proposed circuit. Examples of such requirements are: the precision of the operation, the ease of manufacturing, the efficiency of heat dissipation and the expected production cost. The basic idea, similar to those seen in the previous paragraph, is the association of a mathematical genotype to the circuit scheme: a linear string of symbols, translated in numbers, resumes the topology of the circuit connections. Figure 3.15 gives a rough idea of how this is possible: a number is attributed to a conducting wire, and a symbol coding a component consists of a letter indicating the type of component followed by two or more numbers indicating the wires to which it is connected. A resistor (R) or a capacitor (C) has two connections, while for a transistor (T) three connections need to be specified. This code is proposed only as an indication to the reader, and does not match those used in real applications. Besides, in a real circuit additional numerical values need to be specified for some components, namely the resistance of a resistor or the capacitance of a capacitor. Drawing by the author. Figure 3.15. A genetic code for electronic circuits. The next step is to realize a computer scheme fit to check a large number of variations of genotypes in a search for optimal performances. The algorithm used, the so-called genetic algorithm, is even closer to biology than Palmiter’s model considered before. The production of a new scheme is based on the reproduction of a couple composed from two already present schemes. The genetic material of both parents is split in two parts each, and two halves, one from each parents, are used to buildup a new genetic sequence which will be attributed to their offspring. As a result of this procedure a new circuit is produced with a new connection scheme, but somehow resembling both parents. In a few generations circuits with the most strange and unpredictable structure are present in the populations, while the constraints  Complex Systems and Creativity 21 imposed by the programmer for the survival of the fittest ensure that the variations proposed are potentially useful. The secret is not to impose very strict constraints in order to give a chance to the random changes to discover new possibilities. What is most impressive in the schemes produced by these simulations is not the accuracy with which sometimes they are able to meet the aforementioned requirements, but the fact that, as stated by the researchers themselves (Holland, 1992), sometimes much thinking is necessary to understand the way they work. Of course, the circuit produced by the machine will work independently of the ability of the researcher to catch the general pattern of current fluxes in it, not unlike the ability of the circuit board in our laptops to operate independently of the understanding of most of their user, including the present author. The key point is that the solution, the original solution of the problem, has been delegated in this case to a biology-inspired complex pattern of scheme variations, competition, life, death and change, evolving in a computer simulated world. The genetic algorithms are now used to solve analogous combinatorial problems in biochemistry (in particular the protein folding problem), design of new drugs and materials, aerospace engineering, and many other fields (Holland, 1992). The evolved solutions often surprise the researchers in spite of the fact of being produced by a combinatorial approach. Machines inventing creative solutions to practical problems, although in very specialized fields, are a current reality. CONCLUSION To many, creativity is the sign of consciousness. Several scholars, instead, argued in opposition to this assumption. For example, the first chapter of the 1976 J. Jaynes book “The origin of consciousness in the breakdown of the bicameral mind” strongly supports the opposite point that consciousness is not essential for creative thinking. The study of complex systems and complex behaviors arising from self-organization and scheme competition, suggests that the creative process not only is not necessarily associated with an individual’s mind, but actually is the result of a kind of universal dynamics, which acts already in very elementary physical and chemical systems. Even if a general theory for the self-organization of complex systems, like those envisaged by Prigogine and Haken, has still not been formulated, experiments by computer simulation are now accessible to any interested person, provided she or he is willing to learn how to translate in living, virtual worlds, the basic principles. These experiments, even in their most simple form, are very inspiring, and can help much to give a proper perspective on the creativity of Nature and the role of Man in it. BIBLIOGRAPHICAL NOTES 1 . The complexity of a structure can be evaluated from the structure of its Fourier spectrum, which in a genuinely complex case presents several sharp peaks superimposed to a wide continuum spectrum (Schroeder, 1992). This would be the case of a Fourier analysis of  22 Savino Longo the shape in Figure 3.6. A different measure of complexity can be formulated by evaluating the quantity of information necessary to describe the integration between the whole structure and its substructure (Edelman & Tononi, 2001). This last definition can be applied to describe the complexity of a pattern of activity in a system, and not only its shape. 2 . It is intended that the fraction is reduced to its minimal form, e.g. 20/30 is written 2/3. The resonant orbits (n,k small) are soon depleted. The most stable orbits are given by (n/k)ϕ, where n,k are small and ϕ is the golden ratio (Gutzwiller, 1991) and they are related to the long filaments in Figure 3.2. A similar mechanism explains the fine structure of Saturn’s rings, which are periodically perturbed by the planet’s satellites. 3 . The mixing of ink and water discussed above also occurs, typically, through the formation of vortexes, which are clearly visible if the phenomenon is carefully observed; the true molecular (not helped by vortexes) mixing, which can be studied by pouring the ink drop inside the water slowly and keeping the water still, requires several hours to complete. 4 . The fractal dimension D of an object is determined by estimating the quantity of matter M enclosed within a distance R from one of its points. Generally M = cRD where c is a constant, e.g. for a compact, solid body of density ρ, M = 4/3πρR3 (the mass of a filled sphere) therefore D = 3 as expected. In the case of the Sierpiński gasket and similar abstract objects, we must preliminary associate a fictitious mass to any point or filled cell (like the black cells in Figure 3.6). In this specific case the result is D = log 3 / log 2 = 1.5849, and 1<D<2 as anticipated. Details of the calculation procedure are found in (Schroeder, 1992). When the formula M = cRD does not apply, the object is not a fractal, but an even more complex entity. REFERENCES Armenise, I., Capitelli, M., Garcia, E., Gorse, C., Laganà, A. & Longo, S. (1992). Deactivation dynamics of vibrationally excited nitrogen molecules by nitrogen atoms. Effects on non-equilibrium vibrational distribution and dissociation rates of nitrogen under electrical discharges. Chemical Physics Letters, 200, 597-604. Braitenberg, V. (1984). Vehicles: Experiments in Synthetic Psychology. Cambridge: MIT Press/A Bradford Book. Camazine, S., Deneubourg, J.-L., Franks, N.R., Sneyd, J., Theraula, G., & Bonabeau, E. (2003). Self-Organization in Biological Systems. Princeton University Press. Churchland, P.M. (1995). The Engine of Reason, The Seat of the Soul: A Philosophical Journey into the Brain. Cambridge: MIT Press. Dewdney's column (1987, March) Computer Recreations in Scientific American. Dewdney's column (1989, May) Computer Recreations in Scientific American. Dewdney, A. K. (1990). The Magic Machine: A Handbook of Computer Sorcery. New York: W.H. Freeman & Company. Dickerson, R. E., Gray, H. B., & Haight, G. P. (1973). Chemical Principles. San Francisco: Benjamin-Cummings Publishing Company. Eckhard, R. (1987). Stan Ulam, John von Neumann, and the Monte Carlo Method. Los Alamos Science, Special Issue dedicated to Stanislaw Ulam, 15, 131-143. Retrieved from the web site: http://la-science.lanl.gov/lascience15.shtml  Complex Systems and Creativity 23 Edelman, G., & Tononi, G. (2001). A Universe Of Consciousness: How Matter Becomes Imagination. New York: Basic Books. Epstein I., Kustin K., DeKepper P., & Orbán M. (1983). Oscillating chemical reactions. Scientific American, Mar, 112-123. Fang, L., Xian, L.S., & Kiang, T. (1989). Creation of the Universe. Singapore: World Scientific. Floreano, D. (2001). Manuale sulle reti neurali(2nd ed.). Bologna: Il Mulino. Gutzwiller, M.C. (1991). Chaos in Classical and Quantum Mechanics. New York: Springer. Haken, H. (1983-a). Sinergetica. Il segreto del successo della natura. Torino: Boringhieri. Haken, H. (1983-b). Synergetics, an Introduction: Nonequilibrium Phase Transitions and Self-Organization in Physics, Chemistry, and Biology (3rd rev. enl. ed.). New York: Springer-Verlag. Hebb, D. O. (1949). The Organization of Behaviour. New York: Wiley. Holland, J. H. (1992, July). Genetic Algorithms - Computer programs that evolve in ways that resemble natural selection can solve complex problems even their creators do not fully understand. Scientific American, 267, 66-72. Jaynes, J. (1976) The Origin of Consciousness in the Breakdown of the Bicameral Mind. Boston: Houghton Mifflin. Khanna, T. (1990). Foundations of Neural Networks. New York: Addison-Wesley. Koza, J. R., Bennett, F. H., Andre, D. III, Keane, M.A., & Dunlap, F. (1997). Automated Synthesis of Analog Electrical Circuits by Means of Genetic Programming. IEEE Transactions on Evolutionary Computation, 1, 109-128. Minsky, M., & Papert, S. (1969). Perceptrons. Cambridge: MIT Press. Nicolis, G., & Prigogine I. (1989). Exploring Complexity: An Introduction. New York: W.H. Freeman & Company. Prigogine, I. (1997). The End of Certainty. New York: Free Press. Prigogine, I., & Stengers, I. (1986). La nouvelle alliance. Paris: Gallimard. Schroeder, M. (1992). Fractals, Chaos, Power Laws: Minutes from an Infinite Paradise. New York: W.H. Freeman. Searle, J.R. (1994). The Rediscovery of the Mind. Cambridge: MIT Press. Strycker, P. D., Chanover, N., Sussman, M. & Simon-Miller, A. (2006). A Spectroscopic Search for Jupiter's Chromophores. Bulletin of the American Astronomical Society, 38, 498. Swenson, R. (1992). Order, Evolution and natural Law: Fundamental Relations in Complex System Theory. In C. V. Negoita (ed.) Cybernetics and Applied Systems (pp. 125-148). New York: Marcel Dekker. Toomre, A. & Toomre, J. (1973, June). Violent Tides between Galaxies. Scientific American, 229(6), 38-48.