-
Increased model size. With the improved performance of PyTorch, we were able to significantly increase the size of the model without impacting storage. Firetrace now has 6 hidden layers, with 100 neurons in each. This brings the total parameter count to
11201. The model has been trained for6250epochs.Here is a sample set of results from the updated model.
And a video of its performance during training
output.mp4
-
Used Leaky ReLU activation. We suspected with the release of v2.0.0 that the model may have been suffering from the "dying ReLU" problem. As a precautionary measure we decided to move to leaky ReLU and saw marginal improvements.
-
Used weight decay with the AdamW optimiser. This was another precaution, this time to avoid overfitting the model as we scaled it up.
-
The interface now shows orange fire colours, rather than purple.
- The HuggingFace repository is now divergent from the main GitHub one. A commit is manually created with every release.