Garrus: An Adaptive Companion AI for Video Games using Player Modelling and Deep Reinforcement Learning

Abstract

"Currently, commercial video game AI is mostly developed using comparatively conventional methods such as decision trees, finite state machines, etc. While this is sufficient for narrative driven games to provide an immersive gaming experience, in team based collaborative games, most players would not like to have an AI teammate which is developed using these conventional methods in their team as a replacement for a human player. Achieving the optimal balance of play and making the behaviour of the AI agent as close as possible to an actual human player is considered a considerable challenge in the field. Use of machine learning in game AI development is still a novel field of research which houses a variety of sub-fields that are inhibited by performance limitations and the difficulty of adjusting the player experience when using unpredictable AI of this sort.

This research project will develop a novel training model that can be used to develop adaptive companion AI for video games using a behavioural model created from a main player in combination with boosted reinforcement learning that will utilize that player model as an input. The aim is to achieve this using deep reinforcement learning combined with a novel player modelling approach that will use imitation learning. The prototype will be presented in a form of a Unity ML-Agents brain model-based AI agents running on a custom designed Unity environment/scene.

During the tests done using an evaluation function made by the author himself using observations in the developed game environment and scenario, commendable results were achieved for each effectiveness score (Player modelling – 87.5%, Companion adaptation – 72.5%)"