Feedback of delayed rewards in XCS for environments with aliasing states

Chen, Kuang-Yuan and Lindsay, Peter A. (2009). Feedback of delayed rewards in XCS for environments with aliasing states. In: Kevin Korb, Marcus Randall and Tim Hendtlass, Artificial Life: Borrowing from Biology - Proceedings of 4th Australian Conference on Artificial Life ACAL '09. 4th Australian Conference on Artificial Life ACAL '09, Melbourne, Australia, (252-261). 1-4 December 2009. doi:10.1007/978-3-642-10427-5_25


Author Chen, Kuang-Yuan
Lindsay, Peter A.
Title of paper Feedback of delayed rewards in XCS for environments with aliasing states
Conference name 4th Australian Conference on Artificial Life ACAL '09
Conference location Melbourne, Australia
Conference dates 1-4 December 2009
Proceedings title Artificial Life: Borrowing from Biology - Proceedings of 4th Australian Conference on Artificial Life ACAL '09   Check publisher's open access policy
Journal name Lecture Notes in Computer Science   Check publisher's open access policy
Place of Publication Berlin, Germany
Publisher Springer-Verlag
Publication Year 2009
Year available 2009
Sub-type Fully published paper
DOI 10.1007/978-3-642-10427-5_25
ISBN 9783642104268
ISSN 0302-9743
Editor Kevin Korb
Marcus Randall
Tim Hendtlass
Volume 5865
Start page 252
End page 261
Total pages 10
Language eng
Abstract/Summary Wilson [13] showed how delayed reward feedback can be used to solve many multi-step problems for the widely used XCS learning classifier system. However, Wilson's method based on back-propagation with discounting from Q-learning runs into difficulties in environments with aliasing states, since the local reward function often does not converge. This paper describes a different approach to reward feedback, in which a layered reward scheme for XCS classifiers is learnt during training. We show that, with a relatively minor modification to XCS feedback, the approach not only solves problems such as Woodsl but can also solve aliasing states problems such as Littman57, Miya-zakiA and MazeB.
Subjects 080101 Adaptive Agents and Intelligent Robotics
880399 Aerospace Transport not elsewhere classified
E1
Keyword XCS
Learning Classifier Systems
Aliasing states problem
Credit assignment
Maze problems
Q-Index Code C1
Q-Index Status Confirmed Code
Institutional Status UQ

Document type: Conference Paper
Sub-type: Artificial Life: Borrowing From Biology, Proceedings
Collections: 2010 Higher Education Research Data Collection
School of Information Technology and Electrical Engineering Publications
 
Versions
Version Filter Type
Citation counts: TR Web of Science Citation Count  Cited 0 times in Thomson Reuters Web of Science Article
Scopus Citation Count Cited 0 times in Scopus Article
Google Scholar Search Google Scholar
Created: Thu, 22 Apr 2010, 19:28:57 EST by Marie Walker on behalf of School of Information Technol and Elec Engineering