Seems when we run gatsby-plugin-s3 deploy an putBucketWebsite is sent and hardcodes 404.html to be ErrorDocument even if the targeted s3 bucket is not configured to use 404.html as ErrorDocument.
https://github.com/jariz/gatsby-plugin-s3/blob/9365136039514771d03f965d4f240f55fc7722e7/gatsby-plugin-s3/src/bin.ts#L215