What Users Say?
A Money-Making Machine...

Applications
What Do You Want to Forecast Today?

Technical Papers

Screen Shots

Take a Quick Tour

GeneMiner
Genetic Data Miner, Extracts Useful Rules

Technical Papers

Screen Shots

MailSmart - Removes Spams Automatically

1. Background Information

Fuzzy logic has been applied very successfully in many areas where conventional model based approaches are difficult or not cost-effective to implement. However, as system complexity increases, reliable fuzzy rules and membership functions used to describe the system behavior are difficult to determine. Furthermore, due to the dynamic nature of economic and financial applications, rules and membership functions must be adaptive to the changing environment in order to continue to be useful.

The main advantage of using genetic algorithms (GAs) in economic or financial modeling is that they can be synthesized without making use of the detailed, explicit knowledge of the underlying process. However, limited or noisy training data may result in inconsistent, meaningless output. This has been known to be a severe problem of genetic algorithms.

Because of their complementary nature, these two technologies can be integrated in a number of ways to overcome the drawbacks of each other. A GenoFuzzy hybrid system is described in this paper for the purpose of explanation.

In the following paragraphs, the term "antecedent" refers to the proposition following "IF" in a fuzzy rule 

"IF X1 is A1 and X2 is A2 ...THEN Y is B", 

and that following "THEN" is called the consequent. Membership functions are defined to quantify propositions such as "X is A" and "Y is B".

2. Genetic Algorithms

GA is reminiscent of sexual reproduction in which the genes of two parents combine to form those of their children. When it is applied to problem solving, the basic premise is that we can create an initial population of individuals representing possible solutions to a problem we are trying to solve. Each of these individual has certain characteristics that make them more or less fit as members of the population. The most fit members will have a higher probability of mating than the less fit members, to produce offspring that have a significant chance of retaining the desirable characteristics of their parents. This method is very effective at finding optimal or near optimal solutions to a wide variety of problems, because it does not impose many of the limitations required by traditional methods.

It is an elegant generate-and-test strategy that can identify and exploit regularities in the environment, and converges on solutions that were globally optimal or nearly so.

GA have been increasingly applied in Aritificial Neural Network (ANN) design in several ways: topology optimization, genetic training algorithms and control parameter optimization. In ANN topology optimization, GA is used to select a topology (number of hidden layers, number of hidden nodes, interconnection pattern) for the ANN which in turn is trained using some training scheme, most commonly back-propagation. In genetic training algorithms, the learning of a ANN is formulated as a weights optimization problem, usually using the inverse mean squared error as a fitness measure. Many of the control parameters such as learning rate, momentum rate, tolerance level, etc., can also be optimized using GA.

In addition, GA have been used in many other innovative ways, for instance, creating new indicators based on existing ones, selecting good indicators, to evolve optimal trading systems and to complement other techniques such as fuzzy logic.
 

Fig. 1. The various stages of GA.

 There are four stages in the genetic search process: initialization, evaluation and selection, crossover and mutation. In the initialization stage, a population of genetic structures which are randomly distributed in the solution space is selected as the starting point of the search.

 In the second stage, each structure is evaluated using a fitness function and assigned a fitness value. On the basis of their relative fitness values, structures in the current population are selected for reproduction. A stochastic procedure ensures that the expected number of offsprings associated with a given structure s is u(s)/u(P), where u(s) is the observed performance of s and u(P) is the average performance of all structures in the current population. Thus structures with high performance are more likely to be chosen for replication while poor-performing structures are eventually removed from the population. In the absence of other mechanisms, such a selective process would cause the best performing structures in the initial population to occupy an increasingly larger proportion of the population over time.

The selected structures are recombined using a number of crossover operators, with two complementary search functions. First, it provides new points for further testing of rules already present in the population; Secondly, it introduces instances of new rules into the population.

Generally, crossover draws only on the information present in the solutions of the current population in generating new solutions for evaluation. If specific information is missing (due to storage limitation or loss incurred during the selection process of a previous generation), then crossover is unable to produce new structures that contain this piece of information. A mutation operator, which arbitrarily alters one or more components of a selected structure, provides the means for introducing new information into the population. However, mutation functions as a background operator has a very low probability of application. The presence of mutation ensures that the probability of reaching any point in the search space is never zero.

3. A GenoFuzzy Hybrid System

GenoFuzzy hybrid systems combine the advantages of fuzzy systems, which deal with explicit knowledge that can be explained and understood, and genetic algorithms which deal with implicit knowledge that can be acquired by genetic search, which provides a good way to adjust the expert's knowledge and automatically generate additional fuzzy rules, to meet certain specifications and reduce design time and costs. On the other hand, fuzzy logic enhances the generalization capability of a genetic system by providing more reliable output when extrapolation is needed beyond the limits of the training data.
 

3.1. The GenoFuzzy Architecture

The GenoFuzzy system consists of the various components of a traditional fuzzy system, except that each stage is performed by layers of rude nodes, and a genetic search capability is provided to acquire and enhance the system knowledge.

  
Fig. 2. The schematic of a GenoFuzzy System architecture.

The Fuzzification Layer - Each neuron in this layer represents an input membership function of the antecedent of a fuzzy rule. One common method to implement this layer is to express membership functions as discrete points. Thus for a fuzzy rule "IF X1 is A! and X2 is A2 ... THEN Y is B", A's characterize the possibility distribution of the antecedent clause "X is A". Each of the hidden nodes is defined as a fuzzy reference point in the input space. This method can approximate many continuous functions and the degree of error depends very much on the number of discrete points used. Another much better approach is to use a combination of one to two sigmoidal functions and a linear function to represent each membership function in the Fuzzication and Defuzzification layers. The parameters of these neurons can be trained to fine tune the final shape and location of the membership functions. In most designs, the number of neurons in this layer is fixed, but it is possible to add or remove these neurons during training, according to the outputs produced on the training samples.

In Fig.3, the membership grade indicating the certainty of "X1 is High" is 0.6, "X1 is Medium" is 0.4, and "X1 is Low" is 0.0. The output of these membership function neurons are connected to the Fuzzy Rule Layer as specified by the fuzzy rules, using links with fixed weights of unity.

Fig. 3. Implementation of fuzzy rules.

The Fuzzy Rule Layer - It represents the fuzzy rule base and its function is to perform the fuzzy logical operations. Each neuron represents a fuzzy rule such as "IF X1 is A1 and X2 is A2 ... THEN Y is B" and it calculates the certainty of each compound proposition "IF X1 is A1 and X2 is A2 ... " which indicates the goodness of fit, that is, how well the prerequisites of each fuzzy rule are satisfied. The neurons have a linear function, and their output are connected to the Defuzzification Layer by weighted links. The weights of these links represent the relative significance of the rules associated with the neurons. Their values can be preset according to the expert or initialized to be 1.0, and then trained to reflect their actual importance to the output membership functions contained in the Defuzzification Layer.

The Defuzzification Layer - The function of this layer is for rule evaluation. Each neuron in this layer represents a consequent proposition "THEN Y is B" and its membership function can be implemented by combining one or two sigmoidal functions and linear functions. The certainty of each consequent proposition is calculated, and is regarded as the goodness of fit of those fuzzy rules which have the same consequent proposition. The weight of each output link from these nodes represents the center of gravity of each output membership function of the consequent, and is trainable. The final output value is then calculated using the center of gravity method.

3.2. Fuzzy Rule Implementation 

In the GenoFuzzy system illustrated in Fig. 3, a set of trading rules such as:

Rule 1: IF Prime_Rate is HIGH and DJIA is HIGH THEN Dollars_Purchase is LOW

Rule 2: IF Prime_Rate is LOW and DJIA is LOW THEN Dollars_Purchase is HIGH 

can be programmed easily. A more general format is given below:

Rule 1: IF X1 is High and X2 is Low THEN Y is High .................. 0.8

Rule 2: IF X1 is Medium and X2 is High THEN Y is Medium ....0.5

The value at the end of each rule represents the initial weight of the rule, and will be adjusted to its appropriate level at the end of training. If all the rules have the same subject "Y" for the consequent propositions, then only 1 output node is needed. Fig. 3 illustrates a system with three subjects "Y1", "Y2" and "Y3". The output values could be Dollars transacted, credit worthiness, valuation of real estates, market indices, buy/sell signals and etc.

4. Training the GenoFuzzy System

The structure in Fig. 3 can be configured with initial values specified by human experts, and then further tuned by using a genetic search algorithm as follows:

Step 1: Present an input data sample, compute the corresponding output

Step 2: Compute the error between the output(s) and the actual target(s)

Step 3: The connection weights and membership functions are adjusted

Step 4: At a fixed number of epochs, delete useless rule and membership function nodes, and add in new ones

Step 5: IF Success Rate >= Tolerance THEN goto Step 1 ELSE stop.

When the error level drops to below the user-specified tolerance, the final interconnection weights reflect the changes in the initial fuzzy rules and membership functions. If the resulting weight of a rule is close to zero, the rule can be safely removed from the rule base, since it is insignificant compared to others. Also, the shape and position of the membership functions in the Fuzzification and Defuzzification Layers can be fine tuned by adjusting the parameters of the nodes in these layers, during the training process.

5. Conclusion

GenoFuzzy systems offer the precision and search capability of genetic algorithms, and yet are easy to understand like fuzzy systems. Explicit knowledge acquired from experts can be easily incorporated into such a system, and implicit knowledge can be learned from training samples to enhance the accuracy of the output. Furthermore, the modified and new rules can be extracted from a properly evolved GenoFuzzy system, to explain how the results are derived. There are also many other ways to combine genetic and fuzzy techniques, to improve the learning speed, adjust learning and momentum rates, etc. Also, newer technologies can be integrated to further enhance the performance of the hybrid systems.