Support compression of client-side resources

[Dutchman101]

Feature request
The idea is to support compression of client resources, in order to reduce download size for servers with a substantial gamemode, or if their audience typically has low bandwidth / download speeds.

Since resource .zip's can be mixed (server and clientside), i would propose the following implementation:

1) Add resources subfolder [COMPRESSED]
From here, there are multiple possible approaches:

• Server respects compression level in .zip files supplied by server owner (placed in the COMPRESSED folder) and sends .zip to the client as a whole. This would require mtaserver.conf setting to opt-in, with a description to point out resources in that folder are ONLY allowed to have client-side scripts and assets (everything in its meta.xml must have 'client' attribute) and that they otherwise won't be loaded.

• Server automatically selects clientside files from resources inside the COMPRESSED folder, and zips them using a predefined compression level. For its cache (full of automatically compressed .zip files), it will use a newly added subfolder in "resource-cache"

After completing either of the processing methods listed above, the final result is:

2) Server sends compressed .zip files to the client as a whole, and the client, using newly added unpacking procedure (e.g using the existing vendor library unrar), unpacks them all into deathmatch\resources and uses them like regular resources

Benefits
The benefit is in step 2 - for players with slow internet, unpacking locally in most of the cases is faster than downloading the uncompressed size (unless, of course, their CPU is extremely slow for unpacking.. or the ratio between speed and servers with a huge DL messes it up. Although we certainly will benefit the majority of players playing on average servers with a higher than average DL size).

Cons
Considering the above, we may need to add an option into settings: "Allow compressed resources" with pop-out dialog saying "This will reduce bandwidth usage and increase download speed, at expense of CPU usage when joining a server for the first time. Disable if your PC has trouble during the unpacking stage or locks up". If we do that, we'll need a fallback implementation, like while processing the COMPRESSED folder, server also copies all files cleanly into the main "resource-cache" folders (like it already does) anyways. So then, clients that disable it should download using the regular mechanism despite the server owner opt-in to compression.

Additional context
Implementation is subject to revision by ideas from others, i care about the general idea of supporting compression. I only took 5 minutes to think about the suggested implementation above, it may have flaws.

[CrosRoad95]

may speed up process of downloading for example all flags in default admin panel. Very often servers has a lot of files, and even with my fast internet it can download 100KB/s because of amount of small files.

Be sure someone don't make a compressed bomb

[Disinterpreter]

If you want to compress resources better, I suppose we should to consider LZMA2

it may have flaws.

It have less understandable things like what we must do with transfer compressed files through external HTTP. It must contain file with hashes of compressed files which the client will get.

[Pirulax]

Isn't it enough to just transfer the the hash of the zip on the first time?

[qaisjp]

External HTTP servers can already utilise gzip compression.

Are you wanting extra compression for only the internal HTTP server or are you hoping for this to somehow apply to external HTTP servers too?

[fastman92]

Good idea.
The idea of compression was implemented in the FLA, as a part of enhanced IMG archives.
As such I'm gonna provide some data related to this problem here.

Test of archive:
All files from gta3.img, which comes with GTA San Andreas have been compressed separately from each other with specified compression type and archive fully rebuilt.

IMG_FASTMAN92_GTASA_COMPRESSION_TYPE_NONE - 915 MB archive
IMG_FASTMAN92_GTASA_COMPRESSION_TYPE_ZLIB, highest compression level - 533 MB archive
IMG_FASTMAN92_GTASA_COMPRESSION_TYPE_LZ4, highest compression level, LZ4 HC actually - 686 MB archive

[Pirulax]

Well, I'm not quite sure how external HTTP servers do the compression, but, compressing only a few files VS the whole server is quite a big difference, since you wont need to repeat the dictionary over and over again.

[AlexTMjugador]

External HTTP servers can also use Zopfli compression, whose compression speed is slow, but generates a standard DEFLATE stream that usually has significant savings with respect to other DEFLATE encoders.