The Machine Learning Sandbox is an application in which players can stick together their own neural-network controlled creatures using joints and bones to observe and influence their evolutionary training process towards completing the levels.
After opening the application the player is able to select the first level - the top of a high tower in the midst of dense clouds. Using the ingame creature editor, players can stick together their first creature using bones and joints. When the player has selected a creation and clicks on the Begin Evolution button, a cardboard box appears and drops the first generation of creatures into the level. Each creature is controlled by its own - slightly different - neural network, rotating its joints, through which locomotion emerges. Over generations, creatures can be trained to behave in certain ways by feeding them fitness spheres, rewarding their behaviour. The level is completed as soon as a creature reaches the goal flag, unlocking the next level.
I have always been fascinated by the clever solutions neural networks are able to come up with given various kinds of problems. The mysteries of these black boxes sparked the desire in me to learn about the ins and outs of machine learning by training a neural net myself. For this reason I chose to create an application leveraging machine learning in the context of a playful game environment, combining my two loves for game design and algorithms.
During the first month of working on this project I did nothing but read papers and articles about machine learning, trying to get a feel for the current state of the art. Quickly I learned that the field of machine learning is quite extensive and the approaches as to how neural networks are trained differ a lot. After ruling out deep learning algorithms due to performance and viability concerns, I settled on the evolutionary NEAT algorithm (NeuroEvolution of Augmenting Topologies). Since previous implementations of the algorithm were not suitable for my needs, I refactored an existing implementation and used the resulting UnitySharpNEAT project as the foundation for this project.
After successfully getting the joints of a blocky prototype creature to rotate by the means of a NEAT-ly trained neural net, I began writing on a serialization system for creatures. By being able to save and load creatures an ingame creature editor became possible, allowing players to let all sorts of different behaviour emerge just based on different morphologies. During this whole process I was iterating over the UI design, trying to create a clean and straightforward user experience. Tester feedback revealed that players also wanted some sort of clear goal as well as more variety in the environment, which is why I added a few short levels to give them a well communicated objective and a physical playground.
The main challenge of this project has been selecting and successfully making use of the NEAT algorithm, which turned out to be a good fit for this undertaking. For instance, as soon as a creature starts to jump around and gets rewarded with fitness for it, the next generation will have adapted that jumpy behaviour. This has been a big concern for me in the beginning of the project, since I had no idea whether the manual training via distributing fitness would actually feel impactful, but luckily it all worked out. Sometime in the future I will swap the NEAT algorithm for HyperNEAT (Hypercube-based NEAT), which might produce even more coherent and stunning creature locomotion - though I am happy to not have gone down that road for the time being, since implementing HyperNEAT would have been a massive undertaking, which would have impacted the quality of other features.
In retrospect I am quite happy with the final result, which didn’t seem to be too harshly impacted by the midst of the pandemic, although the social aspect of being with other students and having active conversations about one another's projects is something I did miss a lot. Over the course of my bachelor semester I was able to approach machine learning hands on and satisfy my desire to experiment with neural networks. Although I only scratched the surface of what machine learning has to offer, I was able to develop a good understanding of the isolated research area surrounding evolutionary computation. Overall I deem this project a success and plan to keep working on it.