Home
Patent Search
IMT Blog
REGISTER
|
SIGN IN
United States Patent
5148513
Koza , ; et al.
September 15, 1992
Title
Non-linear genetic process for use with plural co-evolving populations
Abstract
A non-linear genetic process for problem solving using co-evolving populations of entities is disclosed. The iterative process of the present invention operates on a plurality of populations of problem solving entities. First, an activated entity in one of the plurality of populations (evolving population) performs, producing a result. The result is assigned a value and the value is associated with the producing entity. The value assigned is computed relative to the performance of the entity in a population different from the evolving population (one of the environmental populations). Next, entities having relatively high associated values are selected from the evolving population. The selected entities perform either crossover or fitness proportionate reproduction. In addition, other operations such as mutation, permutation, define building blocks and editing may be used. Next, the newly created entities are added to the evolving population. Finally, one of the environmental populations switch roles with the evolving population and the process repeats for the new evolving population and the new environmental populations.
Inventors:
Koza; John R.
(Los Altos Hills,
CA
)
, Rice; James P.
(Redwood City,
CA
)
Assignee:
Koza; John R.
(Los Altos Hills,
CA
)
Appl. No.:
584259
Filed:
September 18, 1990
Current U.S. Class:
706/13
Field of Search:
364/513 395/13
U.S. Patent Documents
4479241
October 1984
Buckley
4675829
June 1987
Clemenson
4697242
September 1987
Holland et al.
4821333
April 1989
Gillies
Primary Examiner:
MacDonald; Allen R.
Attorney, Agent or Firm:
Blakely, Sokoloff, Taylor & Zafman
Parent Case Text
This application is a continuation-in-part patent application of co-pending U.S. patent application Ser. No. 07/500,791; filed Mar. 28, 1990 now abandoned; titled Non-Linear Genetic Algorithms For Solving Problems By Finding a Fit Composition Of Functions; which is a continuation-in-part of U.S. patent application Ser. No. 07/196973 now U.S. Pat. No. 4,935,877; filed May 20, 1988; and titled Non-Linear Genetic Algorithms For Solving Problems.
Claims
What is claimed is:
1. In a parallel processing computer system having at least one processor, a memory, and a plurality of populations of programs of various sizes and structures and wherein more than one program can be executed simultaneously, a group of parallel processes for problem solving wherein more than one parallel process of said group of parallel processes can be performed simultaneously, each parallel process of said group of parallel processes comprising the steps of:
(i) designating one of said plurality of populations as the evolving population wherein the remaining populations of said plurality of populations are designated co-evolving environmental populations;
(ii) iterating a series of steps, said series of steps including:
a) assigning a fitness value to a program of said evolving population and associating said fitness value with a corresponding program, said fitness value indicative of the relative fitness of said corresponding program in relation to said co-evolving environmental populations;
b) selecting at least one program from said evolving population using selection criteria, said selection criteria based on said fitness value associated with each said program, said selection criteria preferring each said program having a relatively high associated fitness value over each said program having a relatively low associated fitness value;
c) choosing and performing an operation, wherein said chosen operation is one of the operations of crossover or reproduction;
d) creating, if said operation is crossover, at least one new program by crossover using a group of programs from said evolving population, said group of programs comprising said selected program and at least one other program from said evolving population, such that any new program created by crossover comprises at least a portion of said selected program and at least a portion of said other program, said new program capable of differing in size and structure from said selected program and said other program;
e) reproducing, if said chosen operation is reproduction, said selected program in said evolving population, such that said selected program remains unchanged; and
f) adding said new program to said evolving population; and
(iii) terminating when a solution is found.
2. The process as claimed in claim 1 wherein said step of choosing and performing an operation further comprising the operation of mutation which occurs before said adding step, wherein said selected program is mutated, such that at least one portion of said selected program is replaced by a randomly generated portion to produce a new program having portions of said selected program and randomly generated portions.
3. The process as claimed in claim 1 wherein said step of choosing and performing an operation includes performing one of said operations for each of said parallel processes and all said parallel processes operate on said evolving population.
4. The process as claimed in claim 1 wherein each of said parallel processes operate on a separate sub-population of said evolving population, said process including a step of periodically intermixing sub-populations of said evolving population.
5. The process as claimed in claim 1 wherein said step of choosing and performing an operation includes performing one of said operations for each of said parallel processes and each of said parallel processes operate on a separate sub-population of said evolving population.
6. The process as claimed in claim 1 wherein said fitness value for said program of said evolving population is assigned relative to a combination of one program from each of a plurality of co-evolving environmental populations.
7. The process as claimed in claim 1 wherein said step of choosing and performing an operation further comprising the operation of permutation, such that if said chosen operation is permutation, a step of permutation occurs before said adding step, wherein said selected program is permuted, such that portions of each said selected program are recordered to create at least one new program from said selected program.
8. The process as claimed in claim 1 wherein said step of selecting at least one program further comprises selection criteria based on a probability that is proportional to said value associated with said program.
9. The process as claimed in claim 1 wherein said step of selecting at least one program further comprises selection criteria based on choosing the better of two randomly selected individual programs.
10. The process as claimed in claim 1 wherein initial populations of programs are created by randomly generating programs of various size and structures, said programs consisting of hierarchical programming structures, said hierarchical programming structures consisting of the functions and arguments available for the problem.
11. A computer system for problem solving comprising:
memory means for storing a plurality of populations of programs of various sizes and structures, each said program being a hierarchical arrangement of functions and arguments;
processing means coupled to said memory means for retrieving said programs stored in said memory means, said processing means executing instructions determined by said retrieved programs;
means for designating one of said plurality of populations as the evolving population wherein the remaining populations of said plurality of populations are designated as co-evolving environmental populations;
means for assigning a fitness value to a program of said evolving population and associating said fitness value with a corresponding program, said fitness value indicative of the relative fitness of said corresponding program in relation to said co-evolving environmental populations;
means for selecting at least one program from said evolving population using selection criteria, said selection criteria based on said fitness value associated with each said program, said selection criteria preferring each said program having a relatively high associated fitness value over each said program having a relatively low associated fitness value;
means for choosing and performing an operation, wherein said chosen operation is one of the operations of crossover or reproduction;
means for creating, if said operation is crossover, at least one new program by crossover using a group of programs from said evolving population, said group of programs comprising said selected program and at least one other program from said evolving population, such that any new program created by crossover comprises at least a portion of said selected program and at least a portion of said other program, said new program capable of differing in size and structure from said selected program and said other program;
means for reproducing, if said chosen operation is reproduction, said selected program in said evolving population, such that said selected program remains unchanged;
means for adding said new program to said evolving population; and
means for terminating when a solution is found.
12. The computer system as claimed in claim 11, wherein said means for selecting at least one program from said evolving population using selection criteria further comprising selection criteria based on a probability that is proportional to said fitness value associated with said program.
13. The computer system as claimed in claim 11 wherein said means for choosing and performing an operation further comprising the operation of mutation such that if said chosen operation is mutation, said selected program is mutated, such that at least one portion of said selected program is replaced by a randomly generated portion to produce a new program having portions of said selected program and randomly generated portions.
14. The computer system as claimed in claim 11 wherein said means for selecting further comprises means for removing at least one program having a relatively low associated fitness value when selecting a program having a relatively high associated fitness value.
15. The computer system as claimed in claim 11 wherein said memory means is used to store the status of all selected and removed programs.
16. The computer system as claimed in claim 11 further comprising a plurality of said processing means for performing parallel operations on said plurality of populations of programs.
17. The computer system as claimed in claim 11 wherein said means for choosing and performing an operation further comprising the operation of permutation, such that if said chosen operation is permutation, said selected program is permuted, such that portions of each said selected program are reordered to create at least one new program from said selected program.
18. The computer system as claimed in claim 11 wherein said means for assigning a fitness value assigns a fitness value indicative of the relative fitness of a program in opposing said co-evolving environmental populations.
19. The computer system as claimed in claim 11 wherein said means for assigning a fitness value assigns a fitness value for a program of said evolving population relative to a combination of one program from each of a plurality of co-evolving environmental populations.
20. The computer system as claimed in claim 11 wherein said plurality of populations of programs stored in said memory means is initially created using means for randomly generating programs of various sizes and structures, said means for randomly generating programs coupled to said processing means, said programs consisting of heierarchical arrangements of the functions and arguments available for the problem.
21. In a computer system having at least one processor, a memory, and a plurality of populations of programs of various sizes and structures, a process for problem solving comprising the steps of:
(i) designating one of said plurality of populations as the evolving population wherein the remaining populations of said plurality of populations are designated co-evolving environmental populations;
(ii) iterating a series of steps, said series of steps including:
a) assigning a fitness value to a program of said evolving population and associating said fitness value with a corresponding program, said fitness value indicative of the relative fitness of said corresponding program in relation to said co-evolving environmental populations;
b) selecting at least one program from said evolving population using selection criteria, said selection criteria based on said fitness value associated with each said program, said selection criteria preferring each said program having a relatively high associated fitness value over each said program having a relatively low associated fitness value;
c) choosing and performing an operation, wherein said chosen operation is one of the operations of crossover or reproduction;
d) creating, if said operation is crossover, at least one new program by crossover using a group of programs from said evolving population, said group of programs comprising said selected program and at least one other program from said evolving population, such that any new program created by crossover comprises at least a portion of said selected program and at least a portion of said other program, said new program capable of differing in size and structure from said selected program and said other program;
e) reproducing, if said chosen operation is reproduction, said selected program in said evolving population, such that said selected program remains unchanged; and
f) adding said new program to said evolving population; and
(iii) terminating when a solution is found.
22. The process as claimed in claim 21 wherein said iterating a series of steps is performed simultaneously for more than one program.
23. The process in claim 21 wherein said step of assigning a fitness value comprising:
executing a program of said evolving population to produce a result by performing said hierarchical arrangement of functions in relation to at least one program from at least one co-evolving environmental population; and
assigning a value to said result and associating said value with a corresponding program from said evolving population which produced said result, said value indicative of the relative fitness of said corresponding program in opposing said co-evolving environmental population, said value relative to the results produced by programs of said co-evolving environmental population.
24. The process as claimed in claim 21 wherein said fitness value for said program of said evolving population is assigned relative to a combination of one program from each of a plurality of co-evolving environmental populations.
25. The process as claimed in claim 21 wherein said step of selecting at least one selected program further comprises selection criteria based on a probability that is proportional to said value associated with said program.
26. The process as claimed in claim 21 wherein said step of selecting at least one selected program further comprises selection criteria based on choosing the better of two randomly selected individual programs.
27. The process as claimed in claim 21 wherein said operation of crossover further comprises taking sub-procedures from at least one said selected program and at least one other program to create a new program, said new program is created solely from sub-procedures of said selected program and sub-procedures of said other program, said new program capable of varying in size and structure from said selected program and said other program.
28. The process as claimed in claim 21 wherein said step of choosing and performing an operation further comprising the operation of permutation, such that if said chosen operation is permutation, a step of permutation occurs before said adding step, wherein said selected program is permuted, such that portions of each said selected program are reordered to create at least one new program from said selected program.
29. The process as claimed in claim 21 wherein said step of choosing and performing an operation further comprises the operation of mutation such that if said chosen operation is mutation, a step of mutation occurs before said adding step, wherein said selected program is mutated, such that at least one portion of said selected program is replaced by a randomly generated portion to produce a new program having portions of said selected program and randomly generated portions.
30. The process in claim 21 wherein said step of choosing and performing an operation further comprising an operation of define building block such that if said chosen operation is said define building block operation, a step of define building block occurs before said adding step, wherein a portion of said selected program is replaced by an invocation of a building block function, said building block function being defined as the hierarchical arrangement of functions and arguments originally associated with said selected portion of said selected program.
31. The process in claim 21 wherein said step of choosing and performing an operation further comprising an operation of editing such that if said chosen operation is said editing operation, a step of editing occurs before said adding step, wherein said selected program is edited, such that predetermined editing rules are applied to said selected program to produce a modified hierarchical structure of said selected program.
32. The process as claimed in claim 21 further comprising a step of removing at least one program having a relatively low associated value.
33. The process as claimed in claim 21 wherein said operation of permutation further comprises permuting a program by rearranging the sub-procedures of said program.
34. The process as claimed in claim 21 wherein said operation of permutation further comprises permuting a program by rearranging the arguments of the sub-procedures of said program.
35. The process as claimed in claim 21 wherein said operation of permutation further comprises permuting a program by rearranging the arguments of the sub-procedures of said program and the sub-procedures of said program.
36. The process as claimed in claim 21 wherein said operation of permutation further comprises permuting a program by redistributing the arguments of all the sub-procedures of said program amongst all the sub-procedures, and reordering the sub-procedures of said program.
37. The process as claimed in claim 21 wherein an individual program in said plurality of populations attaining a pre-established value of fitness with respect to solving the problem is designated as the solution to the problem.
38. The process as claimed in claim 21 wherein a set of programs from one of said plurality of populations collectively attaining a pre-established degree of fitness with respect to solving the problem is designated as the solution to the problem.
39. The process as claimed in claim 21 wherein the entirety of one said population of programs is designated as the solution to a problem and wherein each program in said designated population is equally likely to be selected as a solution to be executed for solving the problem.
40. The process as claimed in claim 21 further including steps for providing a quick fitness calculation for evaluating the fitness of a program in relation to the fitness of programs in said co-evolving environmental populations, said steps including:
storing said fitness value assigned in said assigning step, said fitness value being stored in a memory, said storing step being performed after said assigning step; and
retrieving said fitness value stored in said memory, said retrieving step being performed prior to said assigning step if said memory already contains a fitness value associated with an equivalent combination of programs, said retrieving step performed instead of computing a fitness value for said program.
41. The process as claimed in claim 21 wherein the initial plurality of populations of programs are created by randomly generating programs of various sizes and structures, said programs consisting of hierarchical arrangements of the functions and arguments available for the problem.
42. The process as claimed in claim 21 wherein said populations evolve in relation to a global environment.
43. The process as claimed in claim 21 wherein more than one program is activated simultaneously.
44. The process as claimed in claim 21 wherein said assigning step is performed simultaneously for one program of said evolving population in relation to more than one program of said co-evolving environmental population.
45. The process as claimed in claim 21 wherein said assigning step is performed simultaneously for more than one program.
Description
BACKGROUND OF THE INVENTION
1. The Field of the Invention
The field of the invention is that of genetic processes. More specifically, the field is that of non-linear genetic processes using co-evolving populations suitable for solving problems.
2. The Prior Art
The Natural Selection Process in Nature
The natural selection process provides a powerful tool for problem solving. This is shown by nature and its various examples of biological entities that survive and evolve in various environments. In nature, complex combinations of traits give particular biological populations the ability to adapt, survive, and reproduce in their environments. Equally impressive is the complex, relatively rapid, and robust adaptation and relatively good interim performance that occurs amongst a population of individuals in nature in response to changes in the environment. Nature's methods for adapting biological populations to their environment and nature's method of adapting these populations to successive changes in their environments (including survival and reproduction of the fittest) provides a useful model. This model can develop methods to solve a wide variety of complex problems which are generally thought to require "intelligence" to solve.
In nature, a gene is the basic functional unit by which hereditary information is passed from parents to offspring. Genes appear at particular places (called gene "loci") along molecules of deoxyribose nucleic acid (DNA). DNA is a long thread-like biological molecule that has the ability to carry hereditary information and the ability to serve as a model for the production of replicas of itself. All known life forms on this planet (including bacteria, fungi, plants, animals, and human) are based on the DNA molecule.
The so-called "genetic code" involving the DNA molecule consists of long strings (sequences) of 4 possible gene values that can appear at the various gene loci along the DNA molecule. For DNA, the 4 possible gene values refer to 4 "bases" named adenine, guanine, cytosine, and thymine (usually abbreviated as A, G, C, and T, respectively). Thus, the "genetic code" in DNA consists of a long strings such as CTCGACGGT . . .
A chromosome consists of numerous gene loci with a specific gene value (called an "allele") at each gene locus. The chromosome set for a human being consists of 23 pairs of chromosomes. The chromosomes together provide the information necessary to describe one individual human being and contain about 3,000,000,000 genes. These 3,000,000,000 genes constitute the so-called "genome" for one particular human being. Complete genomes of the approximately 5,000,000,000 living human beings together constitute the entire pool of genetic information for the human species. It is known that certain gene values occurring at certain places in certain chromosomes control certain traits of the individual, including traits such as eye color, susceptibility to particular diseases, etc.
When living cells reproduce, the genetic code in DNA is read. Subsequences consisting of 3 DNA bases are used to specify one of 20 amino acids. Large biological protein molecules are, in turn, made up of anywhere between 50 and 500 such amino acids. Thus, this genetic code is used to specify and control the building of new living cells from amino acids.
The organisms consisting of the living cells created in this manner spend their lives attempting to deal with their environment. Some organisms do better than others in grappling with (or opposing) their environment. In particular, some organisms survive to the age of reproduction and therefore pass on their genetic make-up (chromosome string) to their offspring. In nature, the process of Darwinian natural selection causes organisms with traits that facilitate survival to the age of reproduction to pass on all or part of their genetic make-up to offspring. Over a period of time and many generations, the population as a whole evolves so that the chromosome strings in the individuals in the surviving population perpetuate traits that contribute to survival of the organism in its environment.
Prior Art Genetic Algorithms
Genetic algorithms are highly parallel algorithms that transform populations of individual mathematical objects (typically fixed-length binary character strings) into new populations using operations patterned after (1) natural genetic operations such as sexual recombination (crossover) and (2) fitness proportionate reproduction (Darwinian survival of the fittest). Genetic algorithms begin with an initial population of individuals (typically randomly generated) and then iteratively (1) evaluate the individuals in the population for fitness with respect to the problem environment and (2) perform genetic operations on various individuals in the population to produce a new population. John Holland of the University of Michigan presented the pioneering formulation of genetic algorithms for fixed-length binary character strings in Adaptation in Artificial and Natural Systems, by Professor John H. Holland, 1975. Holland established, among other things, that the genetic algorithm is a mathematically near optimal (minimax) approach to adaptation in that it maximizes expected overall average payoff when the adaptive process is viewed as a multiarmed slot machine problem requiring an optimal allocation of future trials given currently available information. Recent work in genetic algorithms and genetic classifier systems can be surveyed in Grefenstette (1985), Grefenstette (1987), Goldberg (1989), Davis (1987), and Schaffer (1989).
In Adaptation in Artificial and Natural Systems, Holland summarizes his research in genetic algorithms and presents an overall mathematical theory of adaptation for both natural and artificial systems. A key part of this book described a "genetic algorithm" patterned after nature's methods for biological adaptation. However, a limitation of this work resides in using fixed length binary strings to represent the population. U.S. Pat. No. 4,697,242 (Holland) and U.S. Pat. No.
4,881,178 (Holland) are examples of processes which use fixed length binary strings with a genetic algorithm.
Empirical studies by various researchers have demonstrated the capabilities of such genetic algorithms in many diverse areas, including function optimization (De Jong 1980), operation of a gas pipeline (Goldberg 1983), and many others reviewed in Goldberg (1989).
In the chapter entitled "An Overview" contained in the 1987 collection Genetic Algorithms and Simulated Annealing, Lawrence Davis and Martha Steenstrup stated, "In all of Holland's work, and in the work of many of his students, chromosomes are bit strings--lists of 0's and 1's." In addition, they continue, "Some researchers have explored the use of other representations, often in connection with industrial algorithms. Examples of other representations include ordered lists (for bin-packing), embedded lists (for factory scheduling problems), variable-element lists (for semiconductor layout), and the representations used by Glover and Grefenstette in this volume."
Some researchers have attempted to solve search and optimization problems using schemes patterned after evolution that employed mutation-plus-save-the-best strategies. Examples are Box (1957), Hicklin (1986), and the 1966 book by Fogel, Owens, and Walsh entitled Artificial Intelligence Through Simulated Evolution. The few results obtained from these efforts were highly specific to particular applications and domains and largely reflect the cleverness of implementation rather than its usefulness as a general technique for achieving adaptive increases in fitness in populations. It is important to note that mutation is not the primary means by which biological populations in nature improve their fitness and it is not the primary means used in the present invention.
Since Holland's 1975 book, Holland and various colleagues have developed a novel application of conventional genetic algorithms called a "classifier system". A classifier system is a group of rules. Each rule consists of a condition part and an action part (i.e. an IF-THEN rule). Both the condition part and action part of each rule are like the individuals in the conventional genetic algorithm in that they are a strings of 0's and 1's of fixed length. In a classifier system, messages (consisting of binary strings) are received from the environment and activate those rules whose conditional part ("IF" part) match the message (binary string) coming in. This activation triggers the action part ("THEN" part) of the rule. The action part of a rule sends out a new message (binary string).
Classifier Systems are described in the 1978 article "Cognitive Systems based on Adaptive Algorithms" (by Holland and Judith S. Reitman) published in Pattern-Directed Inference Systems, edited by D. A. Waterman and Frederick Hayes-Roth; and David E. Goldberg's 1983 dissertation entitled Computer-Aided Gas Pipeline Operations Using Genetic Algorithms and Rule Learning. In classifier systems, credit is assigned to chains of individual rules that are invoked by a credit allocation scheme known as the "bucket brigade". The Holland process is a combination of a classifier system and a "bucket brigade algorithm". A 1987 paper by Cory Fujiki and John Dickinson in Genetic Algorithms and Their Applications: Proceedings of the Second International Conference on Genetic Algorithms, (John J. Grefenstette, 1987) describes a computer program written in LISP for solving the Prisoner's Dilemma using binary strings of fixed length and IF-THEN classifier rules. In addition, Smith (1980, 1983) has placed IF-THEN rules in genetic strings in lieu of individual characters.
We call conventional genetic algorithms "linear" because they manipulate strings (sequences) of characters over a fixed alphabet (typically strings of binary digits 0 and 1). This is in contrast to the "non-linear" situation in which the objects being manipulated are hierarchical expressions consisting of a hierarchical arrangement of functions and arguments.
The reasons for limiting the conventional genetic algorithm to binary strings of fixed length appear in the literature. First, in his 1983 dissertation entitled Computer-Aided Gas Pipeline Operation Using Genetic Algorithms and Rule Learning, David E. Goldberg argues that any binary string of the common fixed length always has an interpretation (via a well-defined representation scheme) to the problem being solved. This might be called the property of being "well defined" and it is a desirable property.
Secondly, if each individual in the population consists of a binary string of fixed length, then the crossover operation will always produce another binary string of fixed length when applied to any two individuals in the population. This might be called a "closure" property and it is also a desirable property. Of course, binary strings of fixed length are not the only way of achieving these desirable properties of closure and being well-defined.
In Adaptation in Natural and Artificial Systems (1975, page 71), Holland argues in favor of strings consisting only of 0's and 1's (i.e. binary strings) in the conventional genetic algorithm on the basis that the number of strings in the search space that are searched automatically using what he calls the "implicit parallelism" of the conventional genetic algorithm is highest when the strings consist only of two possibilities. This point is true; however, it should not be the controlling consideration. For various reasons cited hereinafter, limiting the genetic algorithm to the one dimensional world of linear strings of fixed length (and, in particular, binary strings of fixed length) precludes solving many problems. The field of computer science is replete with other situations where it is highly unrealistic to assume that the size or shape of a problem is known in advance to the solver so that he can use this information to rigidly pre-specify the size and shape of his search in advance.
Using fixed length binary strings in conventional genetic algorithms limits their ability to solve many problems. The following two separate example problems illustrate additional limitations of conventional genetic algorithms.
First, suppose we want a computer to program itself to solve the problem of finding the point at which two intersecting straight lines intersect. The point of intersection of two straight lines is the pair of numbers that satisfy the two linear equations in two variables that represent the lines. Thus, the computer program we are seeking would use the coefficients of the two equations and various mathematical operators (such as multiplication, subtraction, etc.) to produce the desired answer. To make the problem of having a computer learning to program itself more realistic, it is best not to specify in advance the size or shape of the mathematical expression needed to solve the problem. It is also more realistic if the computer had access to various irrelevant inputs and extraneous mathematical operations that might confuse its search to find the solution to the problem.
There is no simple or convenient way to uniquely associate a binary string whose length is predetermined in advance with an arbitrary mathematical expression composed of specified mathematical operations (functions) and arguments. A binary string of length n can only represent 2.sup.n different things (no matter what the representation scheme). No matter how large an n is pre-selected in advance, there are additional mathematical expressions.
Before continuing, it should be emphasized that it is not necessary to represent things of infinite size. Rather, what should be avoided is arbitrarily pre-setting a limit on the size and shape of the things being represented (even though any particular thing will itself be finite in size). In most problems, the size and shape of the solution are not necessarily known in advance. The process of solving the problem should be free to develop proposed solutions without any pre-set limit on the size and shape of the solution.
Even if an arbitrary maximum length specified in advance were acceptable, the method for associating each arbitrary mathematical expression (for example: A * B+C-D * E * F) with a binary string would necessarily obscure the underlying mathematical operations involved. The highly complex method used by Godel in 1931 in his proof of the Incompleteness Theorem is an example of such a method for making this kind of association. Thus, this first example problem highlights the need to be able to represent arbitrary mathematical expressions (involving various functions and arguments) whose length is not arbitrarily limited in advance (rather than merely strings of 0's and 1's of the same fixed length).
It should be noted that if it is assumed that the two straight lines in this problem always intersect, the problem is entirely numerical. However, if the two lines might possibly be parallel, the answer from a computer program to this expanded version of the problem might appropriately be a symbolic response (e.g. "The Equations are inconsistent and the lines are parallel") rather than the numeric location of the point of intersection. This situation can be easily recognized by a computer program by checking to see if a certain computed value (the determinant) is zero. Thus, this expanded version of this first example problem highlights the need occasionally to accommodate symbolic processing and symbolic output from a computer program that normally produces a numeric output.
Second, consider the problem of predicting the future elements of a sequence of numbers from a sampling of early numbers from the sequence. This problem is an example of induction. Induction is the logical process by which one observes specific examples of some process (e.g. "The sun has come up every morning so far during my life") and then "induces" a reasonable underlying rule for the process (e.g. "The sun always comes up in the morning"). In applying inductive reasoning, there is no proof that the result is correct. Nonetheless, the process of induction is very important and indeed lies at the heart of all learning.
In contrast, deduction is the logical process in which one starts with some given premises (or facts) and some deductive rules of inference and then reaches a logical conclusion by repeatedly applying the deductive rules to the original given premises or facts. The sequence of steps used in deduction to reach a conclusion is called the proof.
If one is given a sampling of a sequence of numbers such as 0, 2, 4, 6, 8, 10, 12, 14 it is not difficult to reasonably induce that the next number in the sequence is 16. The number 16 is a reasonable induction because each previous element of the sequence is 2 times the element's position in the sequence (counting the first element as position 0). Note, however, that even elements of this simple numerical sequence cannot be represented with strings whose length has been specified in advance.
More interesting sequences involve more complicated mathematical operations. For example, the 6th element of the sequence 2, 4, 8, 16, 32, can be expressed directly in mathematics as 2 raised to the 6th power (i.e. 64). This sequence can also be expressed in mathematics using a recursion--that is, by defining the 6th element in terms of previous element(s) in the sequence. In this case, the m.sup.th element of the sequence is 2 times element m-1 of the sequence (that is, 2 times 32 is 64).
For some important mathematical sequences of integers, there is no known non-recursive expression for each element of the sequence, and the use of a recursion becomes a necessity, not merely an option. The well-known Fibonacci sequence 1, 1, 2,
3, 5, 8, 13, 21, 34, 55, is constructed by adding the 2 previous elements of the sequence. For example, 8 is the sum of 3 and 5, and 13 is the sum of 5 and 8. In general, the m.sup.th element of the Fibonacci sequence is the sum of element m-1 and element m-2 of the sequence (with the understanding that the first two elements of the sequence are a "default" value of 1).
Thus, the problem of sequence induction highlights the need to be able to represent recursions as well as arbitrary mathematical expressions (involving functions and arguments). It also re-emphasizes the need to be able to represent strings whose length has not been pre-specified in advance.
Many problems are best approached by developing hierarchies in which solutions to sub-problems are manipulated and assembled hierarchically into solutions to the original main problem. In fact, many mathematical problems are solved by first "decomposing" a larger problem into smaller sub-problems. Then, an attempt is made to solve each of the sub-problems. And, finally, the solutions to the sub-problems are asembled into a solution to the original problem. The problem of solving large numbers of equations with many variables and solving polynomial equations of high order are examples of problems where decomposition can be used. In some cases, there is a symmetry between this process of assembly and the solution to the individual sub-problems. That is, in this assembly process, the solutions to the sub-problems may be manipulated as if they themselves were merely the elements of a sub-problem.
Even when no symmetry is involved, a "hierarchy" develops when a problem is solved by decomposition. At the lowest level of the hierarchy, the sub-problem is solved. The hierarchy consists of combining the solutions of the sub-problem into the solution to the larger problem. Something similar is commonplace in computer programming in general. For example, sub-routines (or sub-procedures) are typically called by a main program. The main program is at the top of the hierarchy, typically organized to provide an overview of the solution to the whole problem. Each of the sub-routines called by the main program are found at one level lower on the hierarchy. If one of the sub-routines itself happens to call upon another sub-routine, that second sub-routine is one level lower on the hierarchy than the sub-routine which called it. Complex social organizations (such as corporations and military organizations), are similarly organized into hierarchies. The ability to decompose problems into hierarchies of sub-problems is generally important for solving problems.
What is needed is a way to apply some of the general principles of biological natural selection that are embodied in the conventional genetic algorithm (i.e. survival of the fittest and crossing over of parent's traits to offspring) to a greatly expanded class of problems. In particular, what is needed is a method for adaptively creating computer programs involving complicated combinations of mathematical functions and their arguments, recursions, symbolic processing, and complicated data structures with no advance limitations on the size, shape, or complexity of the programs. One object of the present invention is to provide a genetic process to provide solutions for an expanded class of problems. A further object of the present invention is to provide a genetic process without any predetermined limits on the size, shape, or complexity of the members of the subject population.
It should be noted, however that the conventional genetic algorithm imposes at least five important limitations which restrict its usefulness in solving a broad range of problems.
First, the requirement that each individual in the population be a string of the same length arbitrarily limits consideration to only a pre-determined number of situations, cases, or states of the problem environment.
Secondly, the use of a binary string (a string of 0's and 1's) leads to a representation scheme involving an explosively large number of "different" solutions merely to handle consideration of only a few past populations. In contrast, if the representation scheme were not required to be rigidly structured in advance prior to the start of operation of the conventional genetic algorithm, a representation scheme involving only a relative handful of relevant possible histories might have evolved.
Thirdly, the individuals in the population are representational descriptions (codings) of a solution (as opposed to being actionable procedures which directly implement the solution). Any particular solution that one envisions and wants to include in the population must be first coded into a binary string of fixed length before it can be inserted into the population. Before any solution can be implemented, the binary string must be decoded into actionable instructions.
Fourthly, the binary strings of fixed length provide no hierarchical structure for potential solutions to the problem. The binary string is one dimensional. All items in the string operate at the same level.
Fifth, it is often true that conventional genetic algorithms are extremely efficient in searching large, complex, non-linear spaces to find an area that is especially good, but that other search techniques are better than conventional genetic algorithms in zeroing in on the final, precise, global optimum value in the search space. Thus, for some problems, it is common to use conventional genetic algorithms to quickly find the best neighborhood of the overall search space and then to switch to another search technique (such as simulated annealing or hill-climbing) to zero in on the precise global optimum value. This shortcoming of conventional genetic algorithms is, for many problems, the direct result of the fixed representation scheme selected at the beginning of the process. If the representation scheme were adaptive (i.e. not fixed), it could change its size and shape after getting into the right general neighborhood of the solution. It could then become more refined so that it would be capable of finding the precise global optimum solution to the problem.
Background on Genetic Programming Paradigm
Representation is a key issue in genetic algorithm work because genetic algorithms directly manipulate the coded representation of the problem and because the representation scheme can severely limit the window by which the system observes its world. Fixed length character strings present difficulties for some problems--particularly problems in artificial intelligence where the desired solution is hierarchical and where the size and shape of the solution is unknown in advance. The need for more powerful representations has been recognized for some time (De Jong 1985, De Jong 1987, De Jong 1988).
The structure of the individual mathematical objects that are manipulated by the genetic algorithm can be more complex than the fixed length character strings. Smith (1980, 1983) departed from the early fixed-length character strings by introducing variable length strings, including strings whose elements were if-then rules (rather than single characters). Holland's introduction of the classifier system (1986) continued the trend towards increasing the complexity of the structures undergoing adaptation. The classifier system is a cognitive architecture into which the genetic algorithm is embedded so as to allow adaptive modification of a population of string-based if-then rules (whose condition and action parts are fixed length binary strings).
In addition, we have recently shown that entire computer programs can be genetically bred to solve problems in a variety of different areas of artificial intelligence, machine learning, and symbolic processing (Koza 1989, 1990). In this recently developed "genetic programming" paradigm, the individuals in the population are compositions of functions and terminals appropriate to the particular problem domain. The set of functions used typically includes arithmetic operations, mathematical functions, conditional logical operations, and domain-specific functions. Each function in the function set must be well defined for any element in the range of every other function in the set which may appear as an argument to that function. The set of terminals used typically includes inputs (sensors) appropriate to the problem domain and various constants. The search space is the hyperspace of all possible compositions of functions that can be recursively composed of the available functions and terminals. The symbolic expressions (S-expressions) of the LISP programming language are an especially convenient way to create and manipulate the compositions of functions and terminals described above. These S-expressions in LISP correspond directly to the "parse tree" that is internally created by most compilers.
The basic genetic operations for the genetic programming paradigm are fitness proportionate reproduction and crossover (recombination). Fitness proportionate reproduction is the basic engine of Darwinian reproduction and survival of the fittest and operates for genetic programming paradigms in the same way as it does for conventional genetic algorithms. The crossover operation for genetic programming paradigms is a sexual operation that operates on two parental programs (i.e. LISP S-expressions) and produces two offspring S-expressions using parts of each parent. In particular, the crossover operation creates new offspring S-expressions by exchanging sub-trees (i.e. sub-lists) between the two parents. Because entire sub-trees are swapped, this genetic crossover (recombination) operation produces syntactically and semantically valid LISP S-expressions as offspring regardless of which allowable point is selected in either parent.
This genetic algorithm paradigm has been successfully applied (Koza 1989, 1990) to example problems in several different areas, including, but not limited to, (1) machine learning of functions (e.g. learning the Boolean 11-multiplexer function), (2) planning (e.g. developing a robotic action sequence that can stack an arbitrary initial configuration of blocks into a specified order), (3) automatic programming (e.g. discovering a computational procedure for solving pairs of linear equations, solving quadratic equations for complex roots, and discovering trigonometric identities), (4) sequence induction (e.g. inducing a recursive computational procedure for the Fibonacci and the Hofstadter sequences), (5) pattern recognition (e.g. translation-invariant recognition of a simple one-dimensional shape in a linear retina), (6) optimal control (e.g. centering a cart and balancing a broom on a moving cart in minimal time by applying a "bang bang" force to the cart), (7) symbolic "data to function" regression, symbolic "data to function" integration, and symbolic "data to function" differentiation, (8) symbolic solution to functional equations (including differential equations with initial conditions, integral equations, and general functional equations), (9) empirical discovery (e.g. rediscovering Kepler's Third Law, rediscovering the well-known econometric "exchange equation" MV=PQ from actual time series data for the money supply, the velocity of money, the price level, and the gross national product of an economy), and (10) simultaneous architectural design and training of neural networks.
Co-evolution in Nature
The evolutionary process in nature is often described as if one population of individuals is trying to adapt to a fixed environment. This simplified description is, however, only a first order approximation to the actual situation. The environment actually consists of both the physical global environment (which is usually relatively unchanging) as well as other independently-acting biological populations of individuals which are simultaneously trying to adapt to "their" environment. The actions of each of these other independently-acting biological populations (species) usually affect all the others. In other words, the environment of a given species includes all the other biological species that contemporaneously occupy the physical environment and which are simultaneously trying to survive. In biology, the term "co-evolution" is sometimes used to reflect the fact that all species are simultaneously co-evolving in a given physical environment.
A biological example presented by Holland illustrates the point (1990). A given species of plant may be faced with an environment containing insects that like to eat it. To defend against its predators (and increase its probability of survival), the plant may, over a period of time, evolve a tough exterior that makes it difficult for the insect to eat it. But, over a period of time, the insect may evolve a stronger jaw so that the insect population can continue to feed on the plant (and increase its probability of survival). Then, over an additional period of time, the plant may evolve a poison to help defend itself further against the insects. But, then again, over a period of time, the insect may evolve a digestive enzyme that negates the effect of the poison so that the insect population can continue to feed on the plant.
In effect, both the plant and the insects get better and better at their respective defensive and offensive roles in this "biological arms race". Each species changes in response to the actions of the other.
Co-evolution and Genetic Processes
In the "genetic algorithm," described by John Holland in his pioneering book Adaptation in Natural and Artificial Systems (1975), a population of individuals attempts to adapt to a fixed "environment." In the basic genetic algorithm as described by Holland in 1975, the individuals in the population are fixed-length character strings (typically binary strings) that are encoded to represent some problem in some way. In the basic "genetic algorithm", the performance of the individuals in the population is measured using a fitness measure which provides information from the "environment" in the form of payoff. Over a period of many generations, the genetic algorithm causes the individuals in the population to adapt in a direction that is dictated by the fitness measure (its environment).
Some work has been done to extend Holland's 1975 genetic algorithm to co-evolutionary situations.
Holland (1990) has incorporated co-evolution and genetic algorithms in his ECHO system for exploring the co-evolution of artificial organisms described by fixed-length character strings (chromosomes) in a "miniature world." In ECHO, there is a single population of artificial organisms. The environment of each organism and the physical global environment includes all other organisms.
Miller (1988, 1989) has used co-evolution in a genetic algorithm to evolve a finite automaton as the strategy for playing the Repeated Prisoner's Dilemma game. Miller's population consisted of strings (chromosomes) of 148 binary digits to represent finite automata with 16 states. Each string in the population represented a complete strategy by which to play the game. That is, it specified what move the player was to make for any sequence of moves by the other player. Miller then used co-evolution to evolve strategies. Miller's co-evolutionary approach contrasts with Alexrod's (1987) solution to the repeated prisoner's dilemma using genetic algorithms. Axelrod measured performance of a particular strategy with a fixed weighted mix of the strategy's results against eight superior opposing computer programs submitted in an international programming tournament for the prisoner's dilemma. A best strategy for one player (represented as a 70 bit string with a 3-move look-back) was then evolved with the weighted mix of eight opposing computer programs serving as the environment.
Hillis (1990) used co-evolution in genetic algorithms to solve optimization problems using fixed length character strings.
What is needed is a way to apply some of the general principles of biological natural selection that are embodied in the conventional genetic algorithm (i.e. survival of the fittest and crossing over of parents' traits to offspring) to co-evolving populations of hierarchical arrangements of functions and arguments that can vary in size and shape. In particular, what is needed is a method for adaptively creating computer programs involving complicated combinations of mathematical functions and their arguments, recursions, symbolic processing, and complicated data structures with no advance limitations on the size, shape, or complexity of the programs. This method for adaptively creating computer programs must be able to operate with the constraint that a best or optimal solution to a problem is not necessarily known. One object of the present invention is to provide a co-evolution genetic process to provide solutions for an expanded class of problems. A further object of the present invention is to provide a co-evolution genetic process without any predetermined limits on the size, shape, or complexity of the members of the subject populations where an optimal solution is not necessarily known.
REFERENCES CITED
U.S. Patents
U.S. Pat. No. 4,821,333, "Machine learning procedures for generating image domain feature detector structuring elements", issued Apr. 11, 1989, filed Aug. 22, 1986, Gillies.
U.S. Pat. No. 4,935,877, "Non-Linear Genetic Algorithms for Solving Problems", issued Jun. 19, 1990, filed May 20, 1988, Koza.
U.S. Pat. No. 4,697,242, "Adaptive Computing System Capable of Learning and Discovery", issued Sep. 29, 1987, filed Jun. 11, 1984, Holland et al.
U.S. Pat. No. 4,881,178, "Method of Controlling a Classifier System", issued Nov. 14, 1989, filed May 7, 1987, Holland et al.
OTHER PUBLICATIONS
Axelrod, Robert (Editor), "The Evolution of Strategies in the Iterated Prisoner's Dilemma" In Genetic Algorithms and Stimulated Annealing, p. 32, Pittman, London 1987.
Binmore, Kenneth G. and Larry Samuelson, "Evolutionary Stable Strategies in Repeated Games Played by Finite Automata" (Draft), Sixth World Congress of the Econometric Society, Barcelona, Spain, Aug. 1990.
Davis, Lawrence (Editor)--Genetic Algorithms and Simulated Annealing, Pitman, London 1987.
De Jong, Kenneth A. Genetic algorithms: A 10 year perspective. Proceedings of an International Conference on Genetic Algorithms and Their Applications. Hillsdale, N.J.: Lawrence Erlbaum Associates 1985.
De Jong, Kenneth A. "On Using Genetic Algorithms to Search Program Spaces", Genetic Algorithms and Their Applications: Proceedings of the Second International Conference on Genetic Algorithms, Hillsdale, N.J.: Lawrence Erlbaum Associates 1987.
De Jong, Kenneth A. Learning with genetic algorithms: an overview. Machine Learning, 3(2), 121-138, 1988.
Fogel, L. J., Owens, A. J. and Walsh, M. J.--Artificial Intelligence through Simulated Evolution, New York: John Wiley 1966.
Fujiki, Cory--An Evaluation of Holland's Genetic Operators Applied to a Program Generator, Master of Science Thesis, Department of Computer Science, University of Idaho, 1986.
Goldberg, David E.--Computer-Aided Gas Pipeline Operation Using Genetic Algorithms and Rule Learning, (Doctoral Dissertation, University of Michigan, 1983) Dissertation Abstracts International 44(10), 3174B (University Microfilms No. 8402282).
Goldberg, David E., Genetic Algorithms in Search, Optimization, and Machine Learning, Reading, Mass.: Addison-Wesley 1989.
Green, Cordell C. et al., Progress Report on Program-Understanding Systems, Stanford Artificial Intelligence Laboratory memo AIM-240, Stanford University Computer Science Department, August 1974.
Grefenstette, John J. (Editor)--Proceedings of an International Conference on Genetic Algorithms and Their Applications, Pittsburgh, Pa. 1985.
Grefenstette, John J. (Editor)--Genetic Algorithms and Their Applications: Proceedings of the Second International Conference on Genetic Algorithms, Lawrence Erlbaum Associates, Hillsdale, N.J. 1987.
Hicklin, Joseph F.--Application of the Genetic Algorithm to Automatic Program Generation, Master of Science Thesis Department of Computer Science, University of Idaho, 1986.
Hillis, W. Daniel, "Co-Evolving Parasites Improve Simulated Evolution as an Optimizing Procedure", Emergent Computation: Self-organizing, Collective, and Cooperative Computing Networks, edited by S. Forrest, Cambridge, Mass., MIT Press 1990.
Holland, John H.--Adaptation in Natural and Artificial Systems, The University of Michigan Press, Ann Arbor, 1975.
Holland, John H. Escaping brittleness: The possibilities of general-purpose learning algorithms applied to parallel rule-based systems. In Michalski, Ryszard S., Carbonell, Jaime G. and Mitchell, Tom M. Machine Learning: An Artificial Intelligence Approach, Volume II. P. 593-623. Los Altos, Calif.: Morgan Kaufman 1986.
Holland, J. H. "ECHO: Explorations of Evolution in a Miniature World." In Proceedings of the Second Conference on Artificial Life, edited by C. G. Langton, and J. D. Farmer, J. Doyne, Redwood City, Calif.: Addison-Wesley. 1990. In Press.
Holland, J. H., & Reitman, J. S. (1978), Cognitive systems based on adaptive algorithms, In D. A. Waterman & F. Hayes-Roth (Eds.), Pattern Directed Inference Systems (pp. 313-329), New York: Academic Press.
Isaacs, Rufus, Differential Games, New York: John Wiley 1965.
Jefferson, David, Collins, Rob, et. al. "The Genesys System: Evolution as a Theme in Artificial Life." In Proceedings of the 11th Iternational Joint Conference on Artificial Life, edited by C. G. Langton and D. Farmer. Redwood City, Calif.: Addison-Wesley. 1990. In Press.
Koza, John R., Genetic Programming: A Paradigm for Genetically Breeding Populations of Computer Programs to Solve Problems, Stanford University, Dept. of Computer Science, Report No. STAN-CS-90-1314, June 1990. 1990.
Koza, John R., Hierarchical genetic algorithms operating on populations of computer programs, Proceedings of the 11th International Joint Conference on Artificial Intelligence (IJCAI); San Mateo, Calif.: Morgan Kaufman 1989.
Lenat, Douglas B. AM: An Artificial Intelligence Approach to Discovery in Mathematics as Heuristic Search, PhD Dissertation, Computer Science Department, Stanford University, 1976.
Lenat, Douglas B., The role of heuristics in learning by discovery: Three case studies, In Michalski, Ryszard S., Carbonell, Jaime G. and Mitchell,
Tom M., Machine Learning: An Artificial Intelligence Approach, Volume I, P. 243-306, Los Altos, Calif.: Morgan Kaufman 1983.
Miller, J. H. "The Evolution of Automata in the Repeated Prisoner's Dilemma." In Two Essays on the Economics of Imperfect Information. PhD dissertation, Department of Economics, University of Michigan, 1988.
Miller, John H., "The Coevolution of Automata in the Repeated Prisoner's Dilemma", Santa Fe Institute and Carnegie-Mellon University, Document No. 89-003, Oct. 15, 1987.
Schaffer, J. D. (editor), Proceedings of the 3rd International Conference of Genetic Algorithms, San Mateo, Calif.: Morgan Kaufman Publishers Inc. 1989.
Smith, Steven F., A Learning System Based on Genetic Adaptive Algorithms, PhD dissertation, Pittsburgh: University of Pittsburg, 1980.
Smith, Steven F., Flexible learning of problem solving heuristics through adaptive search, Proceeding of the 8th International Conference on Artificial Intelligence, Karlsruhe, Germany: Morgan Kaufman 1983.
Tanese, Reiko, Distributed Genetic Algorithm For Function Optimization, PhD. dissertation, Department of Electrical Engineering and Computer Science, University of Michigan, 1989.
Wilson, Stewart, W. Hierarchical credit allocation in a classifier system. Proceedings of the Tenth International Joint Conference on Artificial Intelligence, 217-220, 1987.
SUMMARY OF THE INVENTION
The present invention relates to non-linear genetic processes using co-evolving populations. The process of the present invention operates upon a plurality of co-evolving populations of entities which accomplish tasks. Each entity can vary in size and shape. The process comprises a series of steps including designating an evolving population, activating, assigning, selecting, choosing, creating, reproducing, and adding entities from the designated evolving population. First, one population of the plurality of populations is designated as the evolving population. Next, at least one entity from the evolving population activates to perform its task and produces a result. Next, a value is associated with the result of the activation of each entity and assigned to the corresponding entity. The value associated with the result for the activated entity in the evolving population is computed relative to the result of the activation of a plurality of entities from at least one of the other populations (environmental populations) different from the evolving population. Then, at least one entity in the evolving population having a relatively high associated value is selected. Next, an operation is chosen from crossover or reproduction. In addition, other operations such as mutation, permutation, or "define building blocks" may be used. An editing function may also be used.
If crossover is chosen, then the selected entity (and at least one other entity) performs the crossover operation. Crossover creates new entities by combining portions of at least one selected entity with portions of at least one other entity. Reproduction retains the selected entity in the evolving population. Mutation randomly alters a small random part of an entity. Permutation reorders the parts of an entity without a net gain or loss. Define Building Block replaces portions of an entity with a building block not subject to disruptive operations.
Finally, the newly produced entities are added to the evolving population. One of the plurality of populations (environmental populations) different from the current evolving population is then designated as the evolving population and the process repeats for the new evolving population.
Computer programs have the ability to perform alternative computations conditioned on the outcome of intermediate calculations, to perform computations on variables of many different types, to perform iterations and recursions to achieve the desired result, and to define and subsequently use computed values and sub-programs. This flexibility found in computer programs facilitates the solution to the various problems solved by the present invention.
The process of solving these problems can be reformulated as a search for a most fit individual computer program in the space of possible computer programs. In particular, if the LISP programming language is being used to implement this search, the search space is the hyperspace of LISP "symbolic expressions" (called S-expressions) composed of various terms (called atoms in LISP) along with standard arithmetic operations, standard programming operations, standard mathematical functions, and various functions peculiar to the given problem domain. For example, the standard arithmetic functions of addition, subtraction, multiplication, etc., are relevant when we are attempting to construct a mathematical expression that might be the solution to a differential equation. In general, the objects that are manipulated in our attempts to build computer programs are of four types. These objects include functions of various number of arguments, such as the addition function mentioned above; variable atoms, such as the independent variable(s) in an equation; constant atoms, such as 0, 1, etc.; and control structures such as If-Then-Else, Do-Until, etc.
The LISP S-expression required to solve each of the problems described above tends to emerge from a simulated evolutionary progression using the non-linear genetic process. This process starts with a plurality of initial populations of LISP S-expressions (typically randomly generated) each composed of functions and atoms appropriate to the problem domain.
The fitness of each individual LISP S-expression in each population drives the process. In many problems, fitness can be measured by the sum of the distances or errors (taken for all the cases) between the point in the solution space (whether, for example, real-valued, complex-valued, vector-valued, multiple-valued, Boolean-valued, integer-valued, or symbolic-valued) created by the S-expression for a given set of arguments and the solution created by entities in different populations. In other problems, other fitness measures can be used. The performance of each individual LISP S-expression in an evolving population is thus tested against individual LISP S-expressions from each of the other populations of the plurality of populations. This testing between populations is used to generate the relative fitness for each individual. If fitness is the sum of distances (errors), the closer this fitness sum is to zero, the better the S-expression. Once the desired level of fitness is attained, the iteration of the evolutionary process can be terminated.
The initial individual S-expressions in each population typically will have exceedingly poor fitness. Nonetheless, some individuals in these populations will be somewhat more fit than others.
The process is based on the Darwinian principle of reproduction and survival of the fittest and the genetic operation of crossover (recombination) to create a new population of individuals. In particular, a genetic process of sexual reproduction (crossover) among two parental S-expressions will be used to create offspring S-expressions. At least one of the two participating parental S-expressions will be selected based on fitness. Typically, this selection is proportional to fitness (i.e. fitness proportionate reproduction), although other selection methods (such as tournament selection) may be used. The resulting offspring S-expressions will be composed of sub-expressions from their parents.
In addition, other operations such as mutation and permutation define building blocks and editing may be used.
Finally, the new populations of offspring (i.e. the new generation) will replace the old populations of parents and the process will continue until a desirable solution is found.
At each stage of this highly parallel, locally controlled and decentralized process, the state of the process need consist only of the current populations of individuals. Moreover, the only input to the process will be the observed fitness of the individuals in each current population in grappling with the problem environment.
This process produces populations which, over a period of generations, tend to exhibit increasing average fitness in dealing with the environment consisting of the other populations (and the global environment), and which, in addition, can robustly (i.e. rapidly and effectively) adapt to changes in their environment and other populations.
The solution produced by this process at any given time can be viewed as an entire population of distinctive alternatives (typically with improved overall average fitness). The solution may alternatively be a subset of the population that collectively attains a pre-established degree of fitness (e.g. average level of fitness). More commonly, the solution may be the single best individual in a population at that time ("winner take all"). When the individuals in the population are viewed as alternative strategies (as in a game), the population as a whole may be viewed as a mixed strategy. In this situation, each individual strategy in the population is equally likely to be chosen for execution. That is, the actual play that is executed is the one individual strategy drawn at random from out of the population. Thus, a strategy appearing 10% of the time in the population would be played 10% of the time. Many competitive situations (i.e. games) can only be optimally solved by such a mixed (probabilistic) strategy.
The hierarchical character of the computer programs is an essential aspect of the process. The results of this process are inherently hierarchical and in many cases the results contain default hierarchies which often solve the problem in a relatively parsimonious way.
The dynamic variability of the size and shape of the computer programs that are developed along the way to a solution are also an essential aspect of the process. In each case, it would be difficult and unnatural to try to specify or restrict the size and shape of the eventual solution in advance. Moreover, the advance specification or restriction of the size and shape of the solution to a problem narrows the window by which the system views the world and might well preclude finding the solution to the problem.
DESCRIPTION OF THE DRAWINGS
FIG. 1 is a tree diagram representation of a LISP S-expression.
FIG. 2 is a tree diagram representation of a LISP program.
FIGS. 3A and 3B are flow chart diagrams of the present invention.
FIG. 4 is a tree diagram representation of a crossover operation occurring at internal points.
FIG. 5 is a tree diagram representation of a crossover operation occurring at external points.
FIG. 6 is a tree diagram representation of a crossover operation occurring at an internal and an external point.
FIG. 7 is a tree diagram representation of a permutation operation.
FIG. 8 illustrates a game tree with payoff points.
FIG. 9 illustrates the best and worst strategies in a pursuit game.
FIG. 10 illustrates a sub-optimal strategy in a pursuit game.
FIG. 11 illustrates a typical computer configuration.
FIG. 12 illustrates a simple entity, namely the symbolic expression in the LISP programming language for the mathematical expression A+B*C.
FIG. 13 illustrates the simple entity in FIG. 12 after application of the "Define Building Block" operation.
FIG. 14 illustrates the portion of the simple entity in FIG. 12 being represented by the "Define Building Block" function.
DETAILED DESCRIPTION OF THE INVENTION
The present invention describes a non-linear genetic process for problem solving using co-evolving populations. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. It will be apparent, however, to one skilled in the art that the present invention may be practiced without using these specific details. In other instances, well-known methods and structures have not been described in detail so as not to unnecessarily obscure the present invention.
The present invention operates on a plurality of populations of entities. The entities must possess an ability to produce an objectively observable result. To provide utility, the entities must direct their actions toward a constructive end, even if their results do not always serve those ends. The iterative process of the present invention produces populations which tend to accomplish their constructive ends better than previous populations.
Although the preferred embodiment uses computer programs as entities, using other types of entities remain within the scope and spirit of the present invention. For example, electrical circuits could provide a population for the iterative process of the present invention. The circuits could reproduce and recombine sub-circuits from two parental circuits until a circuit performing the desired behavior (function) is attained. Additionally, different automobile designs could comprise another population, with elements or sub-processes of the designs taken as different alleles for crossover and rearrangement. Thus although the following description uses computer programs as entities, the description does not limit the present invention. Further, the use of sequential iteration is only a preferred embodiment. Methods for the use of parallel processing is also presented.
The Representation of Populations
The computer languages FORTRAN, COBOL, ALGOL, PL/1, FORTH, PASCAL, C, PROLOG, ADA, BASIC, etc. provide, in general, the ability to write complicated mathematical expressions, recursions, complex data structures, and symbolic expressions. Using any of these languages, one can write symbolic expressions that are executable as computational procedures (or programs) within the language itself. Also, any of these languages can generate symbolic expressions, although often this process is inconvenient and inefficient. In general, most computer languages do not allow arbitrarily complex expressions to be written. Also, most do not delay assigning actual computer memory (and types) in the computer for such expressions until just prior to actual execution of the expression. Such a memory management method is termed dynamic storage allocation.
One existing computer language, however, has all the features discussed above and is generally available in relatively efficient forms on a variety of computers. This language is LISP, and is the computer language of choice for many artificial intelligence applications. Many dialects of the LISP language have been created over the years. A dialect of LISP called "Common LISP" has started to emerge as a standard.
The LISP programming language's basic structure is a list of items (an ordered set of items contained within a pair of parentheses). An important source of LISP's simplicity, generality, and power arises from treating the first element in every list encountered as a function to be executed, termed "evaluated", and treating the remaining elements of the list as arguments to that function. Moreover, unless otherwise indicated, LISP reads, evaluates, and returns a value for each such function it encounters. Thus, in LISP, entire computer programs can appear as merely functions within functions within functions (often called "compositions" of functions and arguments or more simply a "composition" of functions). Applying functions to arguments as encountered controls the flow of LISP programs. In other words, the control structures in LISP are based on composition of functions.
Within the outermost pair of parentheses in LISP, there may be numerous functions and operators, including functions for performing arithmetic, functions for performing recursions, functions for modifying symbolic expressions, functions for conditionally varying the program flow, and other complex functions. A key feature of LISP is that LISP programs have the same form as the data they manipulate. As the above features indicate, LISP is a functional programming language. LISP is not the only existing functional programming language nor is it the only possible functional programming language. It is, however, the most widely used language in this category and well-suited for the requirements at hand.
In spite of the complex results obtained, LISP can be viewed as being very simple because it simply reads, evaluates, and returns a value for each such function it encounters. This seeming simplicity gives LISP enormous flexibility (including the flexibility to accommodate computational procedures which modify themselves and execute themselves). This enormous flexibility makes LISP the preferred computer language for the present invention.
For example, consider the simple mathematical expression ordinarily written as 5 * 4-3 * 2. To evaluate this expression, one must start by first evaluating 5 * 4. One evaluates 5 * 4 by performing the function of multiplication (*) on the two arguments (5 and 4). The basic structure in LISP is a list of items (that is, an ordered set of items contained within a set of parentheses). Moreover, unless otherwise indicated, LISP treats the first item in every list encountered as a function and the remaining items in the list as arguments to that function. Thus, LISP represents 5 * 4 as (*5 4). Here a function (i.e. the multiplication function denoted by *) is the first item of the list and the two arguments to the function (i.e. the two numbers to be multiplied) follow. Similarly, LISP denotes 3 * 2 as (*3 2). Once these two multiplications are executed (evaluated), the subtraction function then has the two arguments (i.e. 20 and 6). The two values obtained by evaluating these two multiplication functions are treated as arguments to the subtraction function which performs the operation of subtraction, which is (-(*5 4) (*3 2)). Expressions such as (-(*5 4) (*3 2)) in LISP are called symbolic expressions (S-expressions). Here the function of subtraction (-) is performed on the results previously obtained for (*5 4) and (*3 2). When a simple number or variable is used as the argument of a function (such as the 3 or 2 in the multiplication 3 * 2), it is called an "atomic" argument. The contrasting situation occurs with a composition of functions when the argument to one function is itself the result of carrying out an earlier (embedded) function. We can represent increasingly complex mathematical expressions by embedding previous results within new expressions in this manner.
It is helpful to graphically depict a functional programming language's expressions. Functional expressions can be viewed graphically as a tree with labels on the various points of the tree. In particular, any such expression can be viewed as a rooted point-labeled tree in which the internal points of the tree are labeled with functions and the endpoints of the lines radiating downwards from each such internal point is labeled with the arguments to that function. The term "downwards" in connection with root-point labeled trees means extending farther away from the root of the tree. The external points of the tree (sometimes called "leafs") are labeled with the atomic arguments. The root of the tree is the particular internal point labeled with the function executed last. In a LISP S-expression, the first function is the outer-most LISP function (i.e. the function just inside the outermost left parenthesis of the LISP S-expression).
FIG. 1 illustrates this for LISP using the equation 5 * 4-3 * 2. In the ordinary notation of arithmetic shown as equation 100, the function 104 (multiplication) operates on the arguments 102 and 106 (i.e. 5 and 4 respectively) and the function
112 (multiplication) operates on the arguments 110 and 114 (i.e. 3 and 2 respectively). The function 108 (subtraction) then operates on the results of these two functions as its arguments. The function 108 is higher in the hierarchy than the functions
104 and 112.
In FIG. 1, the LISP S-expression 120, (-(* 5 4) (*3 2)) is expressed as the function 124 (multiplication) operating on the arguments 126 (the number 5) and 128 (the number 4) and the function 130 (multiplication) operating on the arguments 132
(the number 3) and 134 (the number 2). The function 122 (subtraction) then operates on the results of these two evaluations.
When presented graphically in FIG. 1, the internal point 150 of the tree 130 with root 140 is labeled with the function of multiplication (*) and the external points 156 and 158 of the tree are labeled with the two arguments to the multiplication function (i.e. 5 and 4 respectively). The arguments to a given function (such as the multiplication function denoted by the internal point 150) are found by following the lines 152 and 154 radiating downwards from the internal point 150. Similarly, the internal point 160 of the tree is labeled with the function of multiplication and the external points of the tree 166 and 168 are labeled with the two arguments to the multiplication function (i.e., 3 and 2, respectively). The arguments to the function
160 are found by following the lines 162 and 164 radiating downwards from the internal point 160. The internal point of the tree 140 is labelled with the subtraction function. The arguments to the subtraction function are found by following the lines
142 and 144 radiating downwards from point 140. These arguments turn out to be the results of the previously performed multiplication operations. Arguments may be found at external points (if they are "atoms") or at internal points (i.e. when the arguments to one function, such as subtraction here at 140, are the result of previous functions). The internal point 140 is the root of the tree and is labeled with the outermost function (subtraction). Internal point 140 is equivalent to point 122 in the LISP S-expression 120 (i.e., the function just inside the outermost left parenthesis of the LISP S-expression).
The advantage of a computer language such as LISP for performing work of this kind derives from the enormous flexibility arising from repeated applications of this very simple basic structure. The functions available in LISP can include functions other than the simple arithmetic operations of multiplication and subtraction. They include more complex mathematical functions such as square roots, exponentiation, etc; program control functions such as PROGN which allow a series of LISP expressions to be performed in succession; recursions (wherein a function refers to itself in the process of evaluating itself); iterative functions (such as DOTIMES) which cause certain functions to be performed repeatedly (typically with differing arguments); conditional functions [which cause specified alternative functions to be performed if some predicate function is (or is not) satisfied]; and symbolic functions which operate on symbols (instead of numbers).
Note, therefore, that by the term "function", we do not limit ourselves to the strict mathematical meaning of the word. For example, a function in a particular problem domain, which we will call MOVE-BLOCK might be viewed as transforming one state space into another. MOVE-BLOCK might cause a block (in a simulated or in the real world) to be moved from one side of a table to the other. This can be viewed as transforming one state space in which the block is on one side of the table into a new state space in which the block is on the other side of the table. Programmers often view this as a process of side-effecting (i.e. changing the values of) state variables. Thus, by "function" we mean any construct that takes zero or more arguments, returns zero or more values, and transforms the arguments and/or side-effects some global or lexical state. Other examples of "function" using this definition could therefore be "+", which takes numbers as its arguments and returns the sum of the arguments, "PRINT" which takes an argument and prints it (to the global environment), "PROGN" which takes program segments as arguments and returns the result of evaluating its last argument after evaluating all of its other arguments in sequence, and so-called non-strict operators such as "IF" which takes program segments as arguments and returns the result of evaluating one of its arguments dependent upon the result of evaluating its "condition" argument. "MOVE-BLOCK" might be, therefore, a function that takes no arguments, returns no values and whose work consists of side-effecting the state space of the problem. One could also view MOVE-BLOCK as a function that takes an old state space as its argument and returns as its value a new and transformed state space. This definition of "function" therefore subsumes, among others, the terms function, operator, procedure, macro, NLAMBDA and Special Form in LISP and in other programming languages.
By way of an example, suppose we want a computer program to begin by printing the symbolic string "HELLO"; then set the variable C to the sum of the variables A and B; and, then print the value of C only when C is greater than 4. In FIG. 2, the LISP S-expression (i.e. program) 700 performs these tasks. The function 701 PROGN allows a series of 3 major steps to be combined together into one program. The first major step of the series involves the function 702 (PRINT) operating on the symbolic string argument 704 ("HELLO"). The second major step involves the function 706 (SETQ) operating on a variable 708 (C) and the result obtained from the function 710 (addition) operating on the arguments 712 (the variable A) and 714 (the variable B). The SETQ function assigs a value (its second argument) to a variable (its first argument). Finally, the third major step involves the conditional function 716 (WHEN) operating on two arguments. The first argument is a predicate function involving the relationship 718 (greater than) operating on the arguments 720 (the variable C) and 722 (the number 4). The second argument is the function 724 (PRINT) operating on the argument 726 (the variable C).
Graphically, this LISP program (S-expression) can be represented as a tree whose internal points are labeled with functions and where the endpoints of the lines radiating downwards from each such internal point is labeled with the arguments to that function. In this graphical representation, one of the internal points is the root of the tree and the root is labeled with the function that appears just inside the first left parenthesis of the LISP S-expression.
Here, the root of the tree 730 is labeled with the function PROGN. The function PROGN has 3 arguments. The 3 lines 732, 734, and 736 radiating downwards from the internal point 730 (the root) correspond to the 3 arguments of PROGN. The first argument of PROGN is function 738, the PRINT function. It is the endpoint of the first line 732 radiating downwards from internal point 730. The function PRINT has one argument 740. In the case of the PRINT function, it has one argument which it prints. In this case, the argument is the symbolic string 740 "HELLO". This string 740 "HELLO" is an atomic argument and appears at an external point (leaf) of the tree.
The second argument of PROGN is function 742, the SETQ function. The function SETQ has two arguments 744 and 746. The second argument of SETQ is itself a function 746 (addition) operating on the two arguments 748 (the variable A) and 750 (the variable B). The two arguments 748 and 750 are the variables A and B (atoms in LISP). They appear at external points (leafs) of the tree. The first argument of SETQ is 744 (the variable C) which is set to the sum of the values of A and B.
The third argument of PROGN is function 752, the WHEN function. The function WHEN has two arguments, 754 and 756. The first argument of the WHEN function is a predicate function 754 (greater than). The predicate function 754> has two arguments 758 (the value of variable C) and 760 (the number 4). The predicate function 754> returns a value of T (for "True") or NIL (for "False") depending on whether its first argument 758 (the variable C) is greater than its second argument 760
(the number 4). The WHEN function executes its second argument 756 (the PRINT function) if its first argument 754 evaluates as T (True). The PRINT function 756 has one argument 762 (the numeric value of the variable C). Note that the PRINT function is flexible; it can accommodate a symbolic argument (such as "HELLO" at 740) or a number (such as the value of variable C at 762).
Although LISP can be run on virtually any computer, it is preferable to use a computer especially designed for performing LISP functions. The Texas Instruments Explorer II computer is particularly advantageous for these purposes because it contains an especially designed microprocessor chip (called the Mega Chip) which performs LISP functions directly. The Mega Chip contains basic microcode that correspond directly to the basic operations of LISP. These include, among others, basic LISP operations for constructing stacks (which, among other things, retain references to repeated calls on functions) and performing other operations peculiar to LISP. A conventional microprocessor chip (such as the Intel 80286 contained in the IBM AT computer) can be programmed to carry out the various LISP functions by applying its generic computer instructions to the requirements of LISP.
Moreover, it is especially advantageous to run LISP programs on computers with large amounts of internal memory because the complex structures that one develops using LISP in applications such as are described here often require large amounts of memory. To the extent that computer memory is not available as internal memory in a given computer, significant inefficiencies in operation result. Since the solution to problems often require complex structures, significant inefficiencies may make the difference between being able to solve the problem or not being able to solve the problem. The preferred embodiment of the present invention uses an Explorer II computer with 32,000,000 bytes of internal memory (32 megabytes). A typical computer configuration is depicted in FIG. 11.
After generating a population of computational procedures, these procedures are executed and a value in the environment involved is assigned to the result of the execution. Thus an important requirement for any implementation of this system is the ability to generate computational procedures (computer programs) and then execute them to produce a result.
Using LISP representations on a computer having sufficient memory, the present invention can solve problems previously intractable under prior art methods. The following sections present a preferred embodiment of the present invention and specific examples of its application. First, the process itself is described. Secondly, two examples are presented showing the operation of the co-evolutionary process with two games: 1) a simple discrete game with 32 payoff points, and 2) a simple game of pursuit involving two populations (two players).
Co-Evolution and the Genetic Programming Paradigm
The process of co-evolution in nature can be combined with the genetic process to produce a "co-evolution process." In this "co-evolution process," there are two or more populations of individuals. These populations of individuals typically are initially created at random. The environment for the first population (environmental population) in the "co-evolution process" consists of a second population. And, conversely, the environment for the second population consists of the first population.
If there are more than two populations, the environment for one population consists of all the other populations. In all cases, there may be a relatively unchanging (or completely unchanging) background global environment that corresponds to the background physical environment in nature.
The co-evolutionary process typically starts with both populations being highly unfit (when measured by an absolute fitness measure, if such a measure exists). Then, the first population tries to adapt to the "environment" created by the second population. Simultaneously, the second population tries to adapt to the "environment" created by the first population.
This process is carried out by testing the performance of each individual in the first population against each individual (or a sampling of individuals) from the second population. We call this performance the "relative fitness" of an individual because it represents the performance of one individual in one population relative to the environment consisting of the entire second population. Then, each individual in the second population is tested against each individual (or a sampling of individuals) from the first population. Even though we describe the evaluation of an individual in a population and the evaluation of the population itself as an iterative process, this does not preclude concurrent execution of individual populations or whole runs.
Even though both populations are initially generally unfit, the variation that is almost always present in an initial random population will means that some individuals have slightly better relative fitness than others. That means that some individuals in each population will have somewhat better performance than others in dealing with the individuals currently in the opposing population or populations.
The operation of reproduction (based on the Darwinian principle of "survival and reproduction of the fittest) can then be applied to each population using the relative fitness of each individual currently in each population. In addition, the operation of genetic recombination (crossover) can also be applied to a pair of parents, at least one of which is selected from each population based on its relative fitness.
Note that this measurement of relative fitness for an individual in co-evolution is not an absolute measure of fitness against an optimal opponent, but merely a relatives measure when the individual is tested against the current opposing population (environmental population). If one population contains boxers who only throw left punches, then an individual whose defensive repertoire contains only defenses against left punches will have high relative fitness. But, this individual will have only mediocre absolute fitness when tested against an opponent who knows how to throw both left punches and right punches (the optimal opponent).
Over a period of time, both populations of individuals will tend to "co-evolve" and to rise to higher levels of performance as measured in terms of absolute fitness. Both populations do this without the aid of any externally supplied absolute fitness measure serving as the environment. In the limiting case, both populations of individuals can evolve to a level of performance that equals the absolute optimal fitness. Thus, the hierarchical co-evolution process is a self-organizing, mutually-bootstrapping process that is driven only by relative fitness (and not by absolute fitness). Note that although we present a co-evolution example wherein fitness is computed solely as a function of performance relative to an opposing population, the present invention does not preclude the use of a global environment as well.
Co-evolution is likely to be especially important in competitive situations (i.e. games) because one almost never has a priori access to an optimal strategy for either player. One therefore encounters a "chicken and egg" situation. In trying to develop an optimal strategy for the first player, one does not have the advantage of having an optimal second player against which to test candidate strategies. In checkers or chess, for example, it is difficult for a new player to learn to play well if he does not have the advantage of playing against a highly competent player.
Processing Logic of the Preferred Embodiment
FIGS. 3A and 3B are flow charts of the processes of the present invention. The process 1300 starts by the step Create Initial Populations 1321 which creates (typically randomly) a number of populations containing a number of programs. One population is designated as an evolving population. The remaining populations are designated as environmental populations 1302. The process then begins to operate upon the designated evolving population. If processing is not complete for the designated evolving population 1303, control drops to decision box 1305 where a termination condition is tested. If the termination test is satisfied (for example, by achieving a known best solution to the problem among the population of individuals, by achieving a certain degree of fitness for the population, etc.), the process terminates at End 1301. Otherwise, the process continues to iterate. If processing for the evolving population is complete at decision box 1303, control drops to processing box 1304 where a new evolving population is designated. Processing then begins for the new evolving population at decision box 1305.
The basic iterative loop of the process for the evolving population begins with the step Execute Each Entity 1306 wherein at least one entity executes. The next step, Assign Values relative to environmental populations and Associate Values with each Entity 1312, involves assigning a value (fitness) to each result produced by execution, and associating the value with the producing entity. After assigning and associating, Remove Entity(s) with relatively low fitness, step 1314 causes the removal of some of the less fit members of the evolving population (the term "entity(s)" used herein refers to the phrase "entity or entities"). Although not essential, step 1314 improves the average fitness and eases memory requirements by keeping the evolving population within reasonable limits. Step 1316, Select Entity with relatively high fitness values, picks at least one entity to use in the following operation. The selected entity(s) have a relatively high fitness value.
Referring to FIG. 3B at step 1318, Choose an Operation to Perform, the process determines which operation to begin. Crossover 1320 and Reproduction 1330 are the basic operations performed; however, Permutation 1340 also plays a role. Optionally, the operation of Mutation 1350 or Define Building Blocks 1360 may be used. Typically, the vast majority of operations are the reproduction and crossover operations.
Note that the same individual may be selected more than once (i.e., replacement is allowed).
It should be recognized that there are numerous slight variations of the overall process possible. Some of these variations can be used as a matter of convenience.
Crossover 1320 requires a group of at least two entities (typically two parents), so second entity(s) are picked to mate with at least one selected entity(s). There are various methods for choosing the second parent or parents. Generally, choosing only relatively high fitness individuals is preferable over choosing randomly. Parents mate by matching selected entity(s) with at least one second picked entity(s). For each mating, a crossover point is separately selected at random from among both internal and external points within each parent at Select Crossover Points 1322. Then newly created entitys are produced at Perform Crossover 1324 from the mating group using crossover. Two parents would typically produce two offspring.
Note also no requirement exists that the evolving population be maintained at a constant size. The version of the crossover operation producing two offspring from two parents has the convenient attribute of maintaining the population at constant size. (Note that the other operations each produce one offspring from one parent so that they too maintain constant population size). On the other hand, if the crossover operation acts on a group of more than two parents, the size of the population may grow. For example, if three parents formed a mating group, each parent would have two crossover points selected for it and there would be 27 possible offspring (3.times.3.times.3). Even if the three offspring equivalent to the three original parents are excluded, there would be 24 possible new offsprings available. In general, if there are N parents, then N-1 crossover points would be selected for each and there would be N.sup.N -N new offspring available. When an operation produces more offspring than parents, then either the population can be allowed to grow or the population can be trimmed back to a desired (presumably constant) size when the next round of fitness proportionate reproduction takes place.
For the operation of Reproduction 1330, the Selected entity(s) remain unchanged. The preferred method for selecting computational procedures for reproduction is to select them with a probability proportional to their normalized fitness. It is also possible to use tournament selection or other methods of success.
If the permutation operation is selected then the process continues at Permutation 1340. A permutation point is selected at random in Select Permutation Point 1342 from among the internal points within the selected individual. Then Perform Permutation 1344 is performed, by reordering the selected entity's sub-procedures, parameters, or both at the permutation point.
If the mutation option is chosen, Mutation 1350 occurs. The location of the mutation is picked in Select Mutation Point 1352 for each Selected entity. Perform Mutation 1354 then randomly generates, for each Selected entity, a portion of an entity and inserts it at the mutation point. The portion inserted is typically a single point, but may be a sub-entity.
If the Define Building Block operation 1360 is chosen, a new function is defined by replacing the sub-tree located at the chosen point by a call to the newly defined function. The body of the newly defined function is the sub-tree located at the chosen point. The newly defined functions can be named DF0, DF1, DF2, DF3, . . . as they are created.
The editing function 1380 recursively applies a pre-established set of editing rules to each S-expression in the population. In all problem domains, if any sub-expression has only constant atoms as arguments, the editing operation will evaluate that sub-expression and replace it with the value obtained. The "define building block" operation and editing function are described in more detail below.
Finally, the newly created entities are inserted into the evolving population at 1370 and the process returns to the termination test 1303.
The first step in the iterative process involves activating an entity from the evolving population. Activation means having the entity attempt to accomplish its goal, producing an objective result. In the preferred embodiment, entities are computer programs, so activation requires executing the programs of the population. The second step in the process assigns a fitness value to the objective result, and associates that fitness value with its corresponding entity. For computer programs, the fitness value is generally a number, or a vector, which reflects the program's execution, although the fitness value could be any symbolic, numeric or structured representation used on a computer, provided they can be ordered.
In general, some of the entities will prove to be better than others when a value is assigned to them after their interaction with the "environment" of the problem (i.e. the environmental populations and, possibly, the global environment). The best value (fitness) may be the lowest number (as is the case here where we are measuring the aggregated deviation between a result and a known perfect solution). In other problems, the best value (fitness) may be the highest number (e.g. scoring direct "hits"). The value (fitness) assigned may be a single numerical value or a vector of values, although it is often more convenient that it be a single numerical value. In many problems, the best value is not known. However, even in such problems, it is known whether lower (or higher) numbers connote better fitness and the best value attained by the process at a given time can be identified.
A useful method for organizing raw fitness values involves normalizing the raw values, then calculating probabilities based on the normalized values. The best raw fitness value is assigned an adjusted fitness of 1, the worst value is assigned an adjusted value of 0, and all intermediate raw values are assigned adjusted values in the range of 0 to 1. The probability of an individual being selected can be determined in one of several ways. One way is that the probability of being selected is determined by the equation: ##EQU1## Where P(i) is the probability of selection for individual i having an adjusted fitness of f.sub.i, and n is the total number of the population. Thus, an individual's probability of being selected equals the individual's adjusted fitness value divided by the sum of all the adjusted fitness values of the population. In this way, the normalized fitness values range P (i) between 0 and 1, with a value of 1 associated with the best fitness and a value of 0
associated with the worst, and the sum of all the probabilities equals 1. Note that fitness proportionate reproduction requires activation of all the entities in the evolving population in order to compute the sum of the adjusted fitness values f.sub.j needed in the above calculation.
Another way of selecting an individual is called "tournament selection". In tournament selection, two individuals are randomly selected from the population; their fitness is compared; and, the better of the two individuals is selected. This "tournament" method of selection requires less computer time and does not require the centralized computation of the sum of the adjusted fitness values f.sub.j. In effect, this method relies upon the relative ranking of the fitness values, rather than their exact numeric values.
However, if computer time and the centralized computation is not a concern, the "fitness proportionate reproduction" method is generally to be preferred.
It may also be desirable to remove individual computation procedures from the evolving population with relatively poor fitness values. In practice, it may also be convenient to defer this activity briefly until a new generation of individuals is created.
It is a key characteristic of this overall process that the new populations of individuals tends to display, over a period of time, increasing average value (fitness) in the environment involved. Moreover, another characteristic of this overall process is that if the environment changes, the new populations of individuals will also tend to display, over a period of time, increasing average value (fitness) in the new environment involved.
At any given time, there is one individual (or more) in every finite population having a single fitness value that is the best amongst that population. Moreover, some environments have a known best fitness value. Examples are when fitness is measured as deviation from a known answer (e.g. a linear equations problem) or number of matches (e.g. a sequence induction problem). Alternatively, a mixed strategy may be used in determining fitness wherein a mix of pure strategies is used instead of a pure optimum strategy.
The present invention's process may occassionally generate an individual whose value (fitness) happens to equal the known best value. Thus, this overall process can produce the best solution to a particular problem. This is an important characteristic of the overall process, but it is only one characteristic. Another important characteristic (and the one which is more closely analogous to nature) is that a population of individuals exists and is maintained which collectively exhibits a tendency to increase their value (fitness) over a period of time. Also, by virtue of the many individuals with good, but not the very best, fitness values the population exhibits the ability to robustly and relatively quickly deal with changes in the environment. Thus, the variety in the population lowers its overall average value (fitness); additionally, the population's variety gives the population an ability to robustly adapt to changes in the environment.
In executing the overall process, it is often convenient to mark the one (or perhaps more) individuals in the population with the best fitness value amongst that population at any given time. Such marked best individuals are then not subject to removal (as parents), but are instead retained in the population from generation to generation as long as they remain the best. This approach prevents loss of the most fit individual in the population and also provides a convenient reference point for analytical purposes. If the problem involved happens to have a known best solution, after a certain number of generations the best individual will often be the known best solution.
The third step involves selecting entities which will be used to perform operations. A number of selection methods exist which tend to select entities of relatively high value. The theoretically most attractive way to select individuals in the population is to do so with a probability proportionate to their fitness values (once so normalized between 0 and 1). Thus, an individual with fitness of 0.95 has 19 times greater chance of being selected than an individual of fitness value 0.05. Occasionally individuals with relatively low fitness values will be selected. This selection will be appropriately rare, but it will occur.
If the distribution of normalized fitness values is reasonably flat, this method is especially workable. However, if the fitness values are heavily skewed (perhaps with most lying near 1.00), then making the selection using a probability that is simply proportionate to normalized fitness will result in the differential advantage of the most fit individuals in the population being relatively small and the operation of the entire process being prolonged. Thus, as a practical matter, selection is done with equal probability among those individuals with relatively high fitness values rather than being made with probability strictly proportionate to normalized fitness. This is typically accomplished by choosing individuals whose fitness lies outside some threshold value. One implementation of this approach is to select a threshold as some number of standard deviations from the mean (selecting for example, all individuals whose fitness is one standard deviation from the mean fitness).
In connection with selection of individuals on the basis of fitness, we use the phrase "relatively high value" herein to connote either selection based on a probability proportionate to normalized fitness (the preferred approach), tournament selection (the time-saving approach), or selection with equal probability among those individuals having fitness values outside some threshold. In practice, choosing individuals from the best half with equal probability is a simple and practical approach, although fitness proportionate selection is the most justified theoretically.
After completing selection, the fourth step requires choosing an operation. The possible operations include crossover, permutation, and reproduction. In addition, mutation and "define building block" operations are available. The preferred operation is crossover, followed by reproduction, and lastly permutation. However, this preference is only a generalization, different preferences may work better with some specific examples. Thus the choice of operations should mainly be the preferred operation; but that choice should remain flexible to allow for solving differing problems.
As will be seen below, the key operation for introducing new individuals into the population is the crossover operation. To illustrate the crossover operation for this example, a group of two individuals is selected from among the population of individual S-expressions having relatively high fitness values, although, it is not necessary to limit the size of the group selected to two. Two is the most familiar case since it is suggestive of sexual reproduction involving a male parent and a female parent. The underlying mathematical process can obtain effective results by "crossing" hereditary information from three or more parents at one time. However, the key advantage of being able to combine traits from different individuals is attained with two parents. In its preferred form, all of the individuals in the group of parents have relatively high fitness values. In its most general form, the requirement is only that at least one of the individuals in the group of parents has a relatively high fitness value. The other parents in the group could be any member of the population. In either case, all mating involves at least one parent with relatively high fitness values.
For purposes of this example problem, assume that a group of two parents with relatively high fitness values has been selected. The group of parents is now used to create two new individuals. FIG. 4 graphically illustrates a simple example of mating two parents to produce two new offspring for the example problem invo