Leela Zero Pytorch

Generating Leela Zero weights via supervised training with PyTorch

Dataset Generation

I used the KGS game records: https://u-go.net/gamerecords/

Leela Zero comes with a tool that turns sgf files into datasets that can be fed into neural networks. See https://github.com/leela-zero/leela-zero#supervised-learning for more details.

1. Download desired sgf files.
2. Concatenate them into one file so that you can feed it into Leela Zero, e.g. cat *.sgf > train.sgf.
3. Feed the concatenated sgf file to Leela Zero, and it will generate a bunch of gzipped files. Divide them appropriately into train, validation and test folders. The default locations are data/train, data/val and data/test from the root of this repo.

Training

1. Clone the repo and cd into the directory.
2. Install leela-zero-pytorch, by running pip install . or pip install -e . (editable mode).
3. Run python -m leela_zero_pytorch.train. Example: python -m leela_zero_pytorch.train +network=small or python -m leela_zero_pytorch.train +network=huge data.train_data_dir=some/dataset/train +pl_trainer.gpus=-1 +pl_trainer.distributed_backend=ddp. You can also use Weights & Biases as your logger by specifying +logger=wandb.
4. Once the training is over, run python -m leela_zero_pytorch.weights. Example python -m leela_zero_pytorch.weights path/to/checkpoint.ckpt weights.txt.

Pretrained Weights

I have trained three networks using the same training data of about 1.3 million positions generated from the KGS game records. The huge network (weights, config) is strongest, followed by the big network (weights, config), and followed by the small network (weights, config). You can use them directly with Leela Zero.