Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Decompression of sharded mag1 precomputed dataset on s3 fails #7093

Closed
frcroth opened this issue May 23, 2023 · 2 comments · Fixed by #7119
Closed

Decompression of sharded mag1 precomputed dataset on s3 fails #7093

frcroth opened this issue May 23, 2023 · 2 comments · Fixed by #7119
Assignees
@frcroth
Labels
backend bug

Comments

@frcroth
Copy link
Member

frcroth commented May 23, 2023 •
edited
Loading

Steps to Reproduce the bug

  1. Explore s3://bossdb-open-data/iarpa_microns/minnie/minnie65/em/
  2. See that there is no histogram!
  3. Log shows "Reading chunk at s3://bossdb-open-data/iarpa_microns/minnie/minnie65/em/8_8_40/10b43.shard failed <~ chunk.decompress.failed <~ Garbage after a valid .gz stream Stack trace: java.io.IOException: Garbage after a valid .gz stream"

Note that for the same dataset hosted on GCS, it does not fail
@frcroth
Copy link
Member Author

frcroth commented May 30, 2023

Problem seems to have been introduced by #7041

@frcroth frcroth self-assigned this Jun 2, 2023
@frcroth
Copy link
Member Author

frcroth commented Jun 2, 2023

Can be fixed by changing the compressor to the existing gunzip implementation. Will look into this.

@frcroth frcroth mentioned this issue Jun 2, 2023
Merged
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@frcroth frcroth

Labels
backend bug
Projects
None yet
Milestone
No milestone
1 participant
@frcroth