How to handle "chunked" content-encoding in Flutter using Dio?

[MyFakeHub]

I am working on a Flutter application that uses Dio to connect to an API and receive a JSON text. I started working and testing on a local instance of the API and everything was well until it was time to deploy the API to a remote server. I started getting this error in from Dio on some requests:

DioException [unknown]: null
Error: FormatException: Unexpected character (at character 1)
!  ���|�?$k��&$e�����xH9�������1C:��)� a��-&I�ćg$���,5j]���1�ơk����...
^

Although these same requests work perfectly in Postman. After some digging, I found out there was a new value in the header of the requests that don't work, this new value was Transfer-Encoding: "chunked".

I tried setting dio.options.responseType = ResponseType.bytes and that worked until I tried to decode the response body.

What should I do to either receive the JSON correctly or decode the response bytes?

[kuhnroyal]

How did you try to decode it and what happened?
Chunked is always binary. You first have to receive all chunks and then probably do an UTF8 decode, after that you should have a JSON string.

[MyFakeHub]

This is the code I used to try to decode the response

Response response = await _dio.get(endpoint, options: Options(responseType: ResponseType.bytes));
const json = json.decode(utf8.decode(response.data));

It might be missing the "receive all chunks" step. Do you have a sample code that demonstrates the full procces?

[kuhnroyal]

Not sure, you can try using ResponseType.stream.

await dio.get<ResponseBody>(
          endpoint,
          options: Options(responseType: ResponseType.stream),
);

// Collect res.data!.stream
res.data!.stream.....

[MyFakeHub]

Thanks for your help. Turns out I was on the wrong track and I asked the wrong question.

[MyFakeHub]

The solution was to use a Brotli transformer to unzip (or decompress) the response data.

The Content-Encoding of the response header was set to br which (I didn't know) means the data was compressed using the Brotli algorithm. I then used this package that automatically transforms the data into JSON. No need to receive the data as bytes or as a stream.

The code looks something like this

dio.transformer = DioBrotliTransformer();
Response response = await dio.get(endpoint); // no need for options: Options(responseType: ResponseType.bytes)