The field of computer game artificial intelligence (AI) has gained considerable attention over the last few years as its advantages to research become well known. Computer games provide complex, rich environments that act as ready-made simulation test-beds and a general feature set that is familiar to game players. In particular, the first person shooter (FPS) genre has grown to be a popular test-bed for exploring AI techniques and creating more realistic synthetic characters. Although there have been vast improvements in commercial game AI over the last few years, it is arguably the single feature that needs the most improvement. Improving game AI is important to provide interesting and challenging opponents for game players. One approach to potentially improve game AI is the use of reinforcement learning (RL) algorithms. Although previous research has highlighted the potential of RL algorithms to produce unique and realistic game agents, the application of such techniques to control the high-level action selection of FPS game agents has not been investigated.
In addition to limited amounts of investigation, RL techniques are interesting to investigate in the FPS domain as they are flexible enough to be combined with interactive training. It is proposed that a combination of RL and interactive training would yield a more complete approach to game agent design. Interactive training is the process of human users teaching synthetic agents how to behave in their environment. Interactive training has been successfully implemented along with RL in simulations of virtual dogs and humans. Although FPS game agents tend to display more complex behaviours than those seen in the interactive training research, it is arguable that training the game agents interactively has the potential to improve the quality of behaviour in game agents. The outcome of this thesis is the technique used to implement the interactive training tool, which has the potential for code reduction, simplified parameter tuning and controlled learning.
The aim of this thesis is to investigate how interactive training can be used to develop FPS game agents. To begin this investigation, it first must be understood how RL can be applied to the complex and continuous space of FPS games. For this investigation, three techniques are implemented and compared in a purpose-built generic FPS environment. The first algorithm is hard-coded and is based on popular techniques used in commercial FPS games. The second algorithm is based on the RL technique Sarsa(λ). The third algorithm is a modified Sarsa(λ) algorithm, which incorporates interactive training. The techniques are implemented and compared based on quantitative and qualitative data to determine the skill level and quality of resultant behaviours respectively. Following this experimentation, a group of commercial FPS game designers will test the interactive training tool to investigate its suitability to the commercial market.