transient "no active session... context deadline exceeded" errors

[mtanida]

We are using buildkit in CI, and lately, we've been seeing a lot of these types of errors:

#39 59.07 error: failed to copy: httpReadSeeker: failed open: failed to authorize: no active session for 946l7u9cqtl3kkur5pifuipi4: context deadline exceeded

I've enabled debugging by running with docker --debug buildx bake --progress=plain --debug ..., but I'm still finding it difficult to understand what is happening.

My questions:

1. Is there a way to extend this timeout
2. Is there a way that I can get more info debug information?

[mtanida]

I'm not sure how to decipher what is happening here. Any help is greatly appreciated!

#40 sha256:e3a0113c18ba7be52a95ac9f00fd156d80afa2699a2ab53fb8b9fb3d13d6a300 53.48MB / 60.81MB 2.0s
#40 sha256:367788114e9b71a21accde812cabd7f1e3bd6affe4408f96607b7b3d44260a39 103.81MB / 138.15MB 14.6s
#40 sha256:e3a0113c18ba7be52a95ac9f00fd156d80afa2699a2ab53fb8b9fb3d13d6a300 60.81MB / 60.81MB 2.2s done
#40 sha256:c34688126523b6c505848daa66f12cbfd2c7ce85f8ec217e1a079fe0966a7054 1.29GB / 1.72GB 51.5s
#40 sha256:a432e6b6711b8e01226534f15ad598a2198f31537b66a0ba65c1d2a2c23a1b96 0B / 1.11kB 0.2s
#40 sha256:a2ae30306d17ed4b8a2241c490d48acfd4094dd1d401a7a0f106a788aa918393 60.82MB / 1.77GB 5.3s
#40 sha256:367788114e9b71a21accde812cabd7f1e3bd6affe4408f96607b7b3d44260a39 111.15MB / 138.15MB 15.9s
#40 sha256:367788114e9b71a21accde812cabd7f1e3bd6affe4408f96607b7b3d44260a39 118.49MB / 138.15MB 17.3s
#40 sha256:c34688126523b6c505848daa66f12cbfd2c7ce85f8ec217e1a079fe0966a7054 1.37GB / 1.72GB 54.3s
#40 sha256:367788114e9b71a21accde812cabd7f1e3bd6affe4408f96607b7b3d44260a39 125.83MB / 138.15MB 18.5s
#40 56.82 error: failed to copy: httpReadSeeker: failed open: failed to authorize: no active session for mnnt6i5dvruxlbj71kvsv9ejy: context deadline exceeded
#40 56.82 retrying in 1s
#40 sha256:367788114e9b71a21accde812cabd7f1e3bd6affe4408f96607b7b3d44260a39 133.17MB / 138.15MB 19.8s
#40 sha256:ef08d7c56641d212e6e1b3f14aeb7139b5544353e2519e056b1823ddbaf866dd 0B / 23.05MB 0.2s
#40 sha256:a2ae30306d17ed4b8a2241c490d48acfd4094dd1d401a7a0f106a788aa918393 124.78MB / 1.77GB 10.4s
#40 sha256:c34688126523b6c505848daa66f12cbfd2c7ce85f8ec217e1a079fe0966a7054 1.46GB / 1.72GB 57.2s
#40 sha256:367788114e9b71a21accde812cabd7f1e3bd6affe4408f96607b7b3d44260a39 138.15MB / 138.15MB 20.6s done
#40 sha256:b4e8220482f22dfa074e762bdc7c2620c21a1419b0ccb5d64f435327095aa219 0B / 442.14MB 0.2s
#40 sha256:c34688126523b6c505848daa66f12cbfd2c7ce85f8ec217e1a079fe0966a7054 1.55GB / 1.72GB 60.0s
#40 61.82 error: failed to copy: httpReadSeeker: failed open: failed to authorize: no active session for mnnt6i5dvruxlbj71kvsv9ejy: context deadline exceeded
#40 61.82 retrying in 1s
#40 sha256:00099aeee0de4526a31ddbb3603649138e4807bd38074cc9c1a6e1025ce13493 0B / 277.45MB 0.2s
#40 sha256:a2ae30306d17ed4b8a2241c490d48acfd4094dd1d401a7a0f106a788aa918393 196.08MB / 1.77GB 15.5s
#40 sha256:b4e8220482f22dfa074e762bdc7c2620c21a1419b0ccb5d64f435327095aa219 0B / 442.14MB 5.3s
#40 sha256:c34688126523b6c505848daa66f12cbfd2c7ce85f8ec217e1a079fe0966a7054 1.64GB / 1.72GB 63.0s
#40 sha256:c34688126523b6c505848daa66f12cbfd2c7ce85f8ec217e1a079fe0966a7054 1.72GB / 1.72GB 65.9s done
#40 sha256:269f30e2051b6b5f53e62d44c4e7d602b0401aa25984f4943fe745c198e77ac5 0B / 3.14MB 0.2s
#40 66.82 error: failed to copy: httpReadSeeker: failed open: failed to authorize: no active session for mnnt6i5dvruxlbj71kvsv9ejy: context deadline exceeded
#40 66.82 retrying in 1s

[tonistiigi]

Is there a way that I can get more info debug information?

Easiest should be to check docker buildx history trace.

Essentially this error means that BuildKit build is managed by multiple separate grpc requests, and in your case the session request either has dropped or there is some network error and no data is moving in that connection. If it doesn't resolve within a specific timeout, then the build gives up.