[BUG] xrecheck may fail with OverlongHeaders

[rvem]

Description

xrefcheck may fail with OverlongHeaders making it impossible to check a given file.

To Reproduce

I got this error when checking a file that had a link to the Notion page.

Expected behavior

Perhaps, it's worth making it possible to configure max header length for responses that xrefcheck is handling.
This should be possible since http-client-0.7.14

Environment

• OS - NixOS
• branch/revision - 59f4fa3

[rvem]

For reference, here are response headers from the page that is causing this error

$ curl -I <redacted>
HTTP/2 200 
date: Wed, 31 Jan 2024 11:13:28 GMT
content-type: text/html; charset=utf-8
set-cookie: AWSALBTG=0CUeKZ4HU8zL0D4WXDwtUJ1CWMpjiSyRA6GyZtFK8EFVhuLnc0b3elkqmF1OYRE42b5NwF+EThYp96vI8Bw8Go5BaxYsYZL7EgM2tGweI91Uo8LoeChTXZ54g5zm745nVGOty8KolcSVAKmWM2hV5Ru2z77BHOgh4a7eU5ejnK/8; Expires=Wed, 07 Feb 2024 11:13:28 GMT; Path=/
x-notion-request-id: 58cf6490-c765-41b6-a29f-f7306c3aa262
document-policy: js-profiling
content-security-policy: script-src 'self' 'unsafe-inline' 'unsafe-eval' https://gist.github.com https://apis.google.com https://cdn.amplitude.com https://api.amplitude.com https://dev-embed.notion.co https://embed.notion.co https://widget.intercom.io https://js.intercomcdn.com https://static.zdassets.com https://api.smooch.io	 https://solve-widget.forethought.ai https://logs-01.loggly.com https://http-inputs-notion.splunkcloud.com https://cdn.segment.com https://analytics.pgncs.notion.so https://o324374.ingest.sentry.io https://checkout.stripe.com https://js.stripe.com https://embed.typeform.com https://admin.typeform.com https://public.profitwell.com https://static.profitwell.com https://js.sentry-cdn.com https://js.chilipiper.com https://platform.twitter.com https://cdn.syndication.twimg.com https://accounts.google.com https://player.vimeo.com https://www.googletagmanager.com https://www.googleadservices.com https://googleads.g.doubleclick.net https://cdn.metadata.io https://platformapi.metadata.io https://api-gw.metadata.io https://d2hrivdxn8ekm8.cloudfront.net https://d1lu3pmaz2ilpx.cloudfront.net https://dvqigh9b7wa32.cloudfront.net https://d330aiyvva2oww.cloudfront.net https://cdn.transcend.io https://cdn01.boxcdn.net https://cdn.sprig.com https://assets.customer.io https://code.gist.build https://www.google.com https://www.gstatic.com https://challenges.cloudflare.com;connect-src 'self' data: blob: https://msgstore.www.notion.so wss://msgstore.www.notion.so ws://localhost:* ws://127.0.0.1:* https://prod-files-secure.s3.us-west-2.amazonaws.com https://notion-emojis.s3-us-west-2.amazonaws.com https://s3-us-west-2.amazonaws.com https://s3.us-west-2.amazonaws.com https://notion-production-snapshots-2.s3.us-west-2.amazonaws.com https://cdn.amplitude.com https://api.amplitude.com https://www.notion.so https://api.embed.ly https://dev-embed.notion.co https://embed.notion.co https://js.intercomcdn.com https://api-iam.intercom.io https://uploads.intercomcdn.com wss://nexus-websocket-a.intercom.io https://ekr.zdassets.com https://ekr.zendesk.com	 https://makenotion.zendesk.com	 https://api.smooch.io	 wss://api.smooch.io	 https://api.forethought.ai https://logs-01.loggly.com https://http-inputs-notion.splunkcloud.com https://cdn.segment.com https://api.segment.io https://analytics.pgncs.notion.so https://api.pgncs.notion.so https://o324374.ingest.sentry.io https://checkout.stripe.com https://js.stripe.com https://cdn.contentful.com https://preview.contentful.com https://images.ctfassets.net https://www2.profitwell.com https://tracking.chilipiper.com https://api.chilipiper.com https://api.unsplash.com https://api.giphy.com/ https://giphy-analytics.giphy.com/ https://media0.giphy.com/ https://media1.giphy.com/ https://media2.giphy.com/ https://media3.giphy.com/ https://media4.giphy.com/ https://media5.giphy.com/ https://media6.giphy.com/ https://media7.giphy.com/ https://media8.giphy.com/ https://media9.giphy.com/ https://media10.giphy.com/ https://boards-api.greenhouse.io https://accounts.google.com https://oauth2.googleapis.com https://player.vimeo.com https://www.googletagmanager.com https://analytics.google.com https://www.googleadservices.com https://googleads.g.doubleclick.net https://region1.google-analytics.com https://region1.analytics.google.com https://www.google-analytics.com https://cdn.metadata.io https://platformapi.metadata.io https://api-gw.metadata.io https://d2hrivdxn8ekm8.cloudfront.net https://d1lu3pmaz2ilpx.cloudfront.net https://dvqigh9b7wa32.cloudfront.net https://d330aiyvva2oww.cloudfront.net https://cdn.transcend.io https://telemetry.transcend.io https://api.statuspage.io https://pgncd.notion.so https://api.statsig.com https://statsigapi.net https://exp.notion.so https://file.notion.so notion://file.notion.so https://api.box.com https://*.mux.com https://api.sprig.com https://storage.googleapis.com https://cdn.sprig.com https://cdn.userleap.com https://track.customer.io https://*.api.gist.build https://*.cloud.gist.build https://api.palette.dev;font-src 'self' data: https://cdnjs.cloudflare.com https://js.intercomcdn.com https://cdn01.boxcdn.net;img-src 'self' data: blob: https: https://images.ctfassets.net https://platform.twitter.com https://syndication.twitter.com https://pbs.twimg.com https://ton.twimg.com https://region1.google-analytics.com https://region1.analytics.google.com https://file.notion.so notion://file.notion.so https://*.mux.com https://track.customer.io;style-src 'self' 'unsafe-inline' https://cdnjs.cloudflare.com https://github.githubassets.com https://js.chilipiper.com https://platform.twitter.com https://ton.twimg.com https://accounts.google.com https://cdn.transcend.io https://cdn01.boxcdn.net https://code.gist.build;frame-ancestors 'self';worker-src 'self' blob:;child-src 'self' blob:;media-src blob: https: http: https://file.notion.so notion://file.notion.so https://*.mux.com;frame-src https: http: https://accounts.google.com https://renderer.gist.build https://code.gist.build https://challenges.cloudflare.com
x-dns-prefetch-control: off
x-frame-options: SAMEORIGIN
strict-transport-security: max-age=31536000; includeSubDomains; preload
x-download-options: noopen
x-content-type-options: nosniff
x-permitted-cross-domain-policies: none
referrer-policy: strict-origin-when-cross-origin
x-xss-protection: 0
surrogate-control: no-store
cache-control: public,max-age=31536000,immutable
pragma: no-cache
expires: 0
origin-trial: Ak0hZZ1qkZrWdy7N5BYiS9a2hLiGbyfVsL6JZzJSN4yX9t+NA688C/h6Suz2U+RqP8p0GGKqzlocZl5eoc8J+A4AAABzeyJvcmlnaW4iOiJodHRwczovL25vdGlvbi5zbzo0NDMiLCJmZWF0dXJlIjoiVW5yZXN0cmljdGVkU2hhcmVkQXJyYXlCdWZmZXIiLCJleHBpcnkiOjE3MDk4NTU5OTksImlzU3ViZG9tYWluIjp0cnVlfQ==
x-amz-id-2: LKx1drMciJF76Jkjcv/SL1zCgUA4u2xSDUeuowj2DCGOTAAIkuPF8H1376z9yRqHAfD17nKBfKnLfNbz60SCBQ==
x-amz-request-id: BEMZX1DR6KWM19H3
last-modified: Tue, 30 Jan 2024 02:35:44 GMT
x-amz-server-side-encryption: AES256
accept-ranges: bytes
vary: Accept-Encoding
cf-cache-status: DYNAMIC
set-cookie: AWSALBTGCORS=0CUeKZ4HU8zL0D4WXDwtUJ1CWMpjiSyRA6GyZtFK8EFVhuLnc0b3elkqmF1OYRE42b5NwF+EThYp96vI8Bw8Go5BaxYsYZL7EgM2tGweI91Uo8LoeChTXZ54g5zm745nVGOty8KolcSVAKmWM2hV5Ru2z77BHOgh4a7eU5ejnK/8; Expires=Wed, 07 Feb 2024 11:13:28 GMT; Path=/; SameSite=None; Secure
set-cookie: notion_browser_id=5460fef1-2aa7-4b82-b075-5ceb0b29ec54; Domain=notion.site; Path=/; Expires=Thu, 30 Jan 2025 11:13:28 GMT; Secure
set-cookie: device_id=746df677-2c89-4060-b3e5-a35a28289ed0; Domain=notion.site; Path=/; Expires=Thu, 30 Jan 2025 11:13:28 GMT; Secure
set-cookie: notion_check_cookie_consent=false; Domain=notion.site; Path=/; Expires=Thu, 01 Feb 2024 11:13:28 GMT; Secure
set-cookie: notion_locale=en-US%2Fautodetect; Domain=notion.site; Path=/; Expires=Thu, 30 Jan 2025 11:13:28 GMT; Secure
set-cookie: NEXT_LOCALE=en-US; Domain=notion.site; Path=/; Expires=Thu, 30 Jan 2025 11:13:28 GMT; Secure
set-cookie: __cf_bm=7touhda1mAXoOgoJAul14Jm1jyANiYJuOqhtlbyNt4o-1706699608-1-AbSS253RsSFqwX1OTgRflTwgpTULcM0V54uuO42rdVUfQp5yx3ew7k8yogtmK3YUTOvxS2bcrAp7Oi/5oQ0CR2M=; path=/; expires=Wed, 31-Jan-24 11:43:28 GMT; domain=.notion.site; HttpOnly; Secure; SameSite=None
server: cloudflare
cf-ray: 84e1868228ac71a6-BEG
alt-svc: h3=":443"; ma=86400

[zamrokk]

same issue with another notion link