A representation of the model using jzliu-100's repo. You have to admit the vissualization of the neural network is pretty beautiful.
Tic Tac Toe TF is a demonstration of ways that a computer may play tic tac toe against a player. It features methods for player vs player and player vs computer. The computer has a number of different strategies it uses featuring random play, algorithmic play, Q-Tabular play and Neural Network play. Another Student and I coded game.py and algorithm.py during the online coding 101 intensive course at Tufts for our project. However I have heavily added to and redesigned the project since optimizing the performance time of the code and adding Q-Tabular and Neural Networks.
Tic Tac Toe TF supports many different ways of running. The project can easily be cloned into repl.it and to switch to different files please edit the .replit file. There is also a preinstalled pipfile and lock file for pipenv for easy and clean dependency installation. Lastly there is a premade .vscode file if you want to start manipulating the code. I would recommend using the pipenv or repl.it for those whom are not concerned with tweaking the code.
This is the core of the whole project. Game.py contains the default tic tac toe game class with methods to play a move, check for winners and check for valid moves and many more. These methods have been largely refactored since the original build in order to speed up execution. The greatest improvment was made by setting methods so that the majority of calculations were only made after an update to the board occurred. The file also contains a versus() methods where two player can play against each other via the console. In order to run the file simply run python3 game.py in your preffered command line terminal.
Algorithm.py contains a subclass of Game called SmartGame. This new class implements a new method smartMove() which determines the algorithms best possible move it can make. It does this by first determining if there are any winning moves availible. If there are not any it simulates every possible game to completion and tallys the number of games that end in a loss for each move and chooses the move with the smallest value. This file also contains a versus() method to play against the algorithm. In order to run the file simply run python3 algorithm.py in your preffered command line terminal.
Reinforced.py creates its own class called SuperGame. This new class creates the backbone of all methods having to do with Q-Tabular reinforcment learning. The method trainPlay() runs through the set number of epochs updating the q value for each board state through feedReward() based on the result. Moves in each game a stimulated by calling makeMove() which chooses the move with the greates Q-Value. The dataBase() method will update the database.txt file with the q-values associated with all of the x-players moves. Finally the versus() method can be used to play against a computer and the user may choose to be X or O. In order to run the file simply run python3 reinforced.py in your preffered command line terminal.
neural.py contains a subclass of Game called NeuralGame which uses a machine learning TF model to predict the next move. The method neuralMove() simply plays converts the board to a binary representation and gets a prediction from the model and plays it. The class also contains a versus() method which plays a game against the computer using neuralMove(). In order to run the file simply run python3 neural.py in your preffered command line terminal.
trainer2.py creates a class called NeuralTic for training the neural network to play the game. Upon initialization a model is made with Dense layers of 27, 36, 36 and 9. It uses a Stochastic gradient descent optimizer and Mean Squared Error loss function. The model is trained by simulating the specified number of epochs in startTrain(). After each game feedReward() is called which determines the reward and then calculates and applys the gradients to the model.
vissModel.py is a small file tate will create a visual representation of the model and its weights. First it converts the models data from tensors to numpy arrays. Next it uses jzliu-100's repo to display the layers of the model and their weights.