Using Random Weights to Train Multilayer Networks of Hard-Limiting Units

Bartlett P.L. and Downs T. (1992) Using Random Weights to Train Multilayer Networks of Hard-Limiting Units. IEEE Transactions on Neural Networks, 3 2: 202-210. doi:10.1109/72.125861


Author Bartlett P.L.
Downs T.
Title Using Random Weights to Train Multilayer Networks of Hard-Limiting Units
Journal name IEEE Transactions on Neural Networks   Check publisher's open access policy
ISSN 1941-0093
Publication date 1992-01-01
Sub-type Article (original research)
DOI 10.1109/72.125861
Volume 3
Issue 2
Start page 202
End page 210
Total pages 9
Subject 1702 Cognitive Sciences
1705 Computer Networks and Communications
1706 Computer Science Applications
1712 Software
1703 Computational Theory and Mathematics
1708 Hardware and Architecture
2207 Control and Systems Engineering
2208 Electrical and Electronic Engineering
2614 Theoretical Computer Science
Abstract A gradient descent algorithm suitable for training multilayer feedforward networks of processing units with hard-limiting output functions is presented. The conventional Backpropagation algorithm cannot be applied to networks whose processing units have hard-limiting input-output characteristics because the required derivatives are not available. However, if the network weights are random variables with smooth distribution functions, the probability of a hard-limiting unit taking one of its two possible values is a continuously differentiable function. In this paper, we use this to develop an algorithm similar to Backpropagation, but for the hard-limiting case. It is shown that the computational framework of this algorithm is similar to standard Backpropagation, but there is an additional computational expense involved in the estimation of gradients. We give upper bounds on this estimation penalty. Two examples are given which indicate that, when this algorithm is used to train networks of hard-limiting units, its performance is similar to that of conventional Backpropagation applied to networks of units with sigmoidal characteristics.
Q-Index Code C1
Q-Index Status Provisional Code
Institutional Status Unknown

Document type: Journal Article
Sub-type: Article (original research)
Collection: Scopus Import - Archived
 
Versions
Version Filter Type
Citation counts: TR Web of Science Citation Count  Cited 16 times in Thomson Reuters Web of Science Article | Citations
Scopus Citation Count Cited 22 times in Scopus Article | Citations
Google Scholar Search Google Scholar
Created: Tue, 28 Jun 2016, 10:22:40 EST by System User