A problem with the current json based checkpoint files is that this results in a lot of relativley small text files. There are standardized binary file formats as Parquet which might be a good alternative
A problem with the current json based checkpoint files is that this results in a lot of relativley small text files.
There are standardized binary file formats as Parquet which might be a good alternative