Improved Access To Sequential Motifs: A Note On The Architectural Bias Of Recurrent Networks

Boden, Mikael and Hawkins, John (2005) Improved Access To Sequential Motifs: A Note On The Architectural Bias Of Recurrent Networks. IEEE Transactions on Neural Networks, 16 2: 491-494. doi:10.1109/TNN.2005.844086

Attached Files (Some files may be inaccessible until you login with your UQ eSpace credentials)
Name Description MIMEType Size Downloads
archbias5.pdf archbias5.pdf application/pdf 94.46KB 237

Author Boden, Mikael
Hawkins, John
Title Improved Access To Sequential Motifs: A Note On The Architectural Bias Of Recurrent Networks
Journal name IEEE Transactions on Neural Networks   Check publisher's open access policy
ISSN 1045-9227
Publication date 2005-01-01
Sub-type Article (original research)
DOI 10.1109/TNN.2005.844086
Open Access Status File (Author Post-print)
Volume 16
Issue 2
Start page 491
End page 494
Total pages 4
Place of publication New York
Publisher Institute of Electrical and Electronics Engineers
Language eng
Subject 280212 Neural Networks, Genetic Alogrithms and Fuzzy Logic
Abstract For many biological sequence problems the available data occupies only sparse regions of the problem space. To use machine learning effectively for the analysis of sparse data we must employ architectures with an appropriate bias. By experimentation we show that the bias of recurrent neural networks - recently analysed by Tino, Cernansky and Benuskova [8], and Hammer and Tino [9, 3] - offers superior access to motifs (sequential patterns) compared to the, in bioinformatics, standardly used feed forward neural networks.
Keyword recurrent neural network
architectural bias
biological sequence
Computer Science, Artificial Intelligence
Computer Science, Hardware & Architecture
Computer Science, Theory & Methods
Engineering, Electrical & Electronic
Recurrent Neural Network
Protein Secondary Structure
References [1] P. Baldi, S. Brunak, P. Frasconi, G. Soda, and G. Pollastri. Exploiting the past and the future in protein secondary structure prediction. Bioinformatics, 15:937-946, 1999. [2] M. Christiansen and N. Chater. Toward a connectionist model of recursion in human linguistic performance. Cognitive Science, 23:157-205, 1999. [3] B. Hammer and P. Tino. Recurrent neural networks with small weights implement definite memory machines. Neural Comp., 15(8):1897-1929, 2003. [4] S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural Computation, 9(8):1735-1780, 1997. [5] J. F. Kolen. Recurrent networks: State machines or iterated function systems? In M. C. Mozer, P. Smolensky, D. S. Touretzky, J. L. Elman, and A. S. Weigend, editors, Proceedings of the 1993 Connectionist Models Summer School, pages 203{210, Hillsdale, NJ, 1994. Erlbaum Associates. [6] G. Pollastri, D. Przybylski, B. Rost, and P. Baldi. Improving the prediction of protein secondary strucure in three and eight classes using recurrent neural networks and profiles. Proteins, 47:228-235, 2002. [7] J. Schmidhuber, F. Gers, and D. Eck. Learning nonregular languages: A comparison of simple recurrent networks and LSTM. Neural Comp., 14(9):2039-2041, 2002. [8] P. Tino, M. Cernansky, and L. Benuskova. Markovian architectural bias of recurrent neural networks. IEEE Transactions on Neural Networks, 15(1):6-15, 2004. [9] P. Tino and B. Hammer. Architectural bias in recurrent neural networks: Fractal analysis. Neural Comp., 15(8):1931-1957, 2003.
Q-Index Code C1
Q-Index Status Provisional Code
Institutional Status UQ
Additional Notes Originally published as Boden, Mikael and Hawkins, John (2005) Improved access to sequential motifs: A note on the architectural bias of recurrent networks. IEEE Transactions on Neural Networks 16(2). Copyright 2005 IEEE.

Document type: Journal Article
Sub-type: Article (original research)
Collections: Excellence in Research Australia (ERA) - Collection
School of Information Technology and Electrical Engineering Publications
Version Filter Type
Citation counts: TR Web of Science Citation Count  Cited 2 times in Thomson Reuters Web of Science Article | Citations
Scopus Citation Count Cited 4 times in Scopus Article | Citations
Google Scholar Search Google Scholar
Created: Mon, 21 Mar 2005, 10:00:00 EST by Mikael Boden on behalf of School of Information Technol and Elec Engineering