This thesis addresses the use of hybrid learning systems that combine supervised and unsupervised learning methods for pattern classification. Most neural network research applied to pattern recognition has been focussed on supervised learning, and network models like the MLP provide an efficient method to design an arbitrarily complex non-linear classifier. However, there are some problem domains that are not solved in a satisfactory way by means of a single classifier. When the abstraction level of the classification task increases, the shape of the decision regions can become very complex, requiring impossibly large amounts of training data to form the class boundaries. This problem can be alleviated by using unsupervised learning techniques to reduce the number of degrees of freedom in the data. Hybrid learning systems which combine supervised and unsupervised learning methods have been very popular in this regard.
This thesis introduces a novel hybrid system with a hierarchical architecture which is based on the neural gas (NG) algorithm for pattern recognition problems. The NG algorithm in the proposed learning system uses a much faster variation of the original NG algorithm by reducing the time complexity of its sequential implementation. The computationally expensive part of the adaptation step of the original NG algorithm is the determination of the neighborhood ranking. This requires an explicit ordering of all distances between the reference vectors and the input pattern, and this has time complexity 0(N log N). This problem is addressed here by introducing an implicit ranking method which reduces the time complexity to 0(N).
The proposed learning system generates multiple classifications for every data pattern presented, and these are registered as "confidence values". The most suitable functional form for calculating confidence values was determined empirically and it smoothly assigns confidence values from 1 to 0. These confidence values allow the system to employ a variety of classifier fusion techniques to combine individual classifications to produce the predicted class for a pattern. Four different classifier combination techniques were used in the comparisons. It was shown that combining a network performance measure with confidence values by means of the fuzzy integral leads to the best classification performance. The performance of the proposed system was compared with that of other techniques on three well-known benchmark data sets, and promising results were obtained.
Finally, it was shown that the boosting algorithm can be applied to a learning system that uses mixed supervised/unsupervised methods. The boosted learning system gave improved results over those obtained without boosting.