| Formatted abstract
|
There are a number of ways to train neural networks, but they can be put into two rough categories: non-constructive and constructive. When training networks using a non-constructive algorithm, such as back-propagation, one is required determine the architecture, and then present it to the algorithm which will then compute the synaptic weights, as best it can. However, determining the architecture required for solving a particular problem is a difficult task, which is usually done by trail-and error. Constructive algorithms, such as the tiling algorithm, attempt to alleviate this difficulty by using various techniques to automatically generate the architecture, while also setting the synaptic weights. They usually work by adding neurons one at a time, iteratively breaking down the problem to manageable parts. The algorithm which is presented in this thesis is a constructive algorithm for the training of multilayer perceptions. However, it differs from most other constructive algorithms in that it generates architectures one layer at a time, as opposed to one neurons at a time. To generate each layer a simple parallel genetic algorithm is utilized. The genetic algorithm uses the concepts of niche formation and co-operative populations. Each individual in the population of the genetic algorithm represents a single neuron with its synaptic weights. The individuals of the population co-operate to break the problem into linearly separable components. Upon convergence, the population has divided itself into a number of subpopulations, each containing one part of the solution. A layer of a multilayer perceptron can then be produced from this population, by selecting a single neuron/individual from each subpopulation. To facilitate training of the next layer, the training data. is fed through the current layer. The method is applied to a. number of benchmarking problems, including, the XOR problem, the parity problems, the MONK's problems, the Sonar problem, and the two spirals problem. It is found that the performance of is largely comparable to that of the back-propagation algorithm, but it has the advantage of automatically generating the architectures required for solving each particular problem.
|
