Training bots to play: Investigating interactive reinforcement learning for bot behaviours in shooter games

McPartland, Michelle (2012). Training bots to play: Investigating interactive reinforcement learning for bot behaviours in shooter games PhD Thesis, School of Information Technol and Elec Engineering, The University of Queensland.

Attached Files (Some files may be inaccessible until you login with your UQ eSpace credentials)
Name Description MIMEType Size Downloads
s41106585_phd_submission.pdf Thesis full text application/pdf 3.60MB 4
Author McPartland, Michelle
Thesis Title Training bots to play: Investigating interactive reinforcement learning for bot behaviours in shooter games
Formatted title
Training Bots to Play: Investigating Interactive Reinforcement Learning for Bot Behaviours in Shooter Games
School, Centre or Institute School of Information Technol and Elec Engineering
Institution The University of Queensland
Publication date 2012
Thesis type PhD Thesis
Supervisor Marcus Gallagher
Total pages 231
Total colour pages 85
Total black and white pages 146
Language eng
Subjects 0803 Computer Software
0899 Other Information and Computing Sciences
080602 Computer-Human Interaction
080399 Computer Software not elsewhere classified
080199 Artificial Intelligence and Image Processing not elsewhere classified
0801 Artificial Intelligence and Image Processing
Formatted abstract
The field of computer game artificial intelligence (AI) has gained considerable attention over the last few years as its advantages to research become well known. Computer games provide complex, rich environments that act as ready-made simulation test-beds and a general feature set that is familiar to game players. In particular, the first person shooter (FPS) genre has grown to be a popular test-bed for exploring AI techniques and creating more realistic synthetic characters. Although there have been vast improvements in commercial game AI over the last few years, it is arguably the single feature that needs the most improvement. Improving game AI is important to provide interesting and challenging opponents for game players. One approach to potentially improve game AI is the use of reinforcement learning (RL) algorithms. Although previous research has highlighted the potential of RL algorithms to produce unique and realistic game agents, the application of such techniques to control the high-level action selection of FPS game agents has not been investigated.

In addition to limited amounts of investigation, RL techniques are interesting to investigate in the FPS domain as they are flexible enough to be combined with interactive training. It is proposed that a combination of RL and interactive training would yield a more complete approach to game agent design. Interactive training is the process of human users teaching synthetic agents how to behave in their environment. Interactive training has been successfully implemented along with RL in simulations of virtual dogs and humans. Although FPS game agents tend to display more complex behaviours than those seen in the interactive training research, it is arguable that training the game agents interactively has the potential to improve the quality of behaviour in game agents. The outcome of this thesis is the technique used to implement the interactive training tool, which has the potential for code reduction, simplified parameter tuning and controlled learning.

The aim of this thesis is to investigate how interactive training can be used to develop FPS game agents. To begin this investigation, it first must be understood how RL can be applied to the complex and continuous space of FPS games. For this investigation, three techniques are implemented and compared in a purpose-built generic FPS environment. The first algorithm is hard-coded and is based on popular techniques used in commercial FPS games. The second algorithm is based on the RL technique Sarsa(λ). The third algorithm is a modified Sarsa(λ) algorithm, which incorporates interactive training. The techniques are implemented and compared based on quantitative and qualitative data to determine the skill level and quality of resultant behaviours respectively. Following this experimentation, a group of commercial FPS game designers will test the interactive training tool to investigate its suitability to the commercial market.
Keyword Computer games
Reinforcement learning
Interactive training
First person shooters

Citation counts: Google Scholar Search Google Scholar
Created: Mon, 14 Oct 2013, 22:07:46 EST by Michelle Mcpartland on behalf of Scholarly Communication and Digitisation Service