A convolutional neural network applied to Need For Speed Most Wanted.
Created using Python, Tensorflow and TFLearn with CNN models based upon Alexnet and GoogLeNet.
Originally created to test viability of autonomous vehicles in video games.
Input into the convolutional neural network is an image screen capture taken every frame using PIL. Output is a one hot-encoded state representing what key should be pressed. This state is then multiplied by a smoothing coefficient and thresholded to get the overall prediction. This prediction is then fed into pyautogui to emulate the keyboard presses.
Network was trained in a similar way. Data was recorded with screen capture and current keypress’ grouped together. Original implementation used numpy arrays for manipulating data but due to limitations in RAM I had to convert the code to handle data using the HDF5 format.
As an initial test 5 viable movement states were used. [W A S D NK] representing Forward, Left, Back, Right and No Key. This proved to be quite successful but lacked the precision I had hoped for. As a result I increased the key states to represent simultaneous key movements [W WA WD S SA SD A D NK] but due to time constraints I had minimal training data (< 50,000 samples).
The results are still somewhat satisfactory for the minimal encoded model. (More data = Better results!) and provides a solid foundation for further progress to be made.
Network trained using Adelaide University Phoenix High Performance Computing Service as Tensorflow does not play nice with AMD GPU’s!!!