Handle zstd encoding

[viginum-datalab]

Issue description

Sometimes, with big responses, I get the following error:

Err: <class 'UnicodeDecodeError'>: 'utf-8' codec can't decode byte 0xb8 in position 1: invalid start byte

Fixes: #185

Investigation

So I opened my web browser and took a look at the headers.

Request:

Accept-Encoding: gzip, deflate, br, zstd

Response:

content-encoding: zstd

Examining the response data shows it was indeed compressed by the zstd algorithm. Because it wasn't handled, it raised a UnicodeDecodeError that was catched by the general exception in queue_client.py line 38.

Solution

I added zstd to the project dependency and decoded the response content depending on its content-encoding header.