Pynwb tutorial documentation bug #149
Comments
|
@MarikeReimer Pillow handles a number of formats, but not .ims. I wouldn't really expect it to, since this appears to be a proprietary sub-format of HDF5 for a specific image acquisition device. It looks like Pillow does support an abstract class that can be used to support custom formats like .ims here. If you want to use Pillow, I would recommend going this route. There are many proprietary image formats for imaging data (see the Bioformats docs for a sampling), and it's beyond the scope of the tutorials to provide guidance for users to read their own varied data formats. I suppose we could add an explicit note about what formats are supported by Pillow. Would that address your issue sufficiently? |
|
Hi Ben,
I tried converting our .ims file to a .tif file with ImageJ, which unfortunately didn't work with Pillow. In my experience, tif files are the most commonly used alternative to proprietary formats for image stackes. If Pillow doesn't handle them, I think your users would benefit from a little more guidance.
Best,
Marike Reimer|Data Science Architect
ph: 206-240-5051
Center for Neuroscience and Regeneration Research
https://medicine.yale.edu/cnrr/
…________________________________
From: Ben Dichter ***@***.***>
Sent: Tuesday, July 16, 2024 4:04 PM
To: dandi/helpdesk ***@***.***>
Cc: Reimer, Marike ***@***.***>; Mention ***@***.***>
Subject: Re: [dandi/helpdesk] Pynwb tutorial documentation bug (Issue #149)
@MarikeReimer<https://github.com/MarikeReimer> Pillow handles a number of formats<https://pillow.readthedocs.io/en/stable/handbook/image-file-formats.html#im>, but not .ims. I wouldn't really expect it to, since this appears to be a proprietary sub-format of HDF5 for a specific image acquisition device. It looks like Pillow does support an abstract class that can be used to support custom formats like .ims here<https://pillow.readthedocs.io/en/stable/handbook/image-file-formats.html#hdf5>. If you want to use Pillow, I would recommend going this route.
There are many proprietary image formats for imaging data (see the Bioformats docs<https://bio-formats.readthedocs.io/en/v7.3.1/supported-formats.html> for a sampling), and it's beyond the scope of the tutorials to provide guidance for users to read their own varied data formats. I suppose we could add an explicit note about what formats are supported by Pillow. Would that address your issue sufficiently?
—
Reply to this email directly, view it on GitHub<#149 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AIPFK7GH5WJKFQ5RL4ZHRLTZMV4EXAVCNFSM6AAAAABJ4PT3HKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMZRG4ZTKMJZGU>.
You are receiving this because you were mentioned.Message ID: ***@***.***>
|
|
Pillow does support TIFF file format, but it looks like you are trying to open a .ims file in these lines: img = Image.open("MyData.ims") # an example image |
|
Hi Ben,
After meeting with Cody, I updated my test script to document that issue - the code linked to the Github issue is out of date. Would you like me to upload the latest version and/or test files?
[cid:8b59c104-d403-4cb5-a852-68cb46de71af]
Best,
Marike Reimer|Data Science Architect
ph: 206-240-5051
Center for Neuroscience and Regeneration Research
https://medicine.yale.edu/cnrr/
…________________________________
From: Ben Dichter ***@***.***>
Sent: Tuesday, July 16, 2024 4:53 PM
To: dandi/helpdesk ***@***.***>
Cc: Reimer, Marike ***@***.***>; Mention ***@***.***>
Subject: Re: [dandi/helpdesk] Pynwb tutorial documentation bug (Issue #149)
Pillow does support TIFF file format<https://pillow.readthedocs.io/en/stable/handbook/image-file-formats.html#tiff>, but it looks like you are trying to open a .ims file in these lines:
img = Image.open("MyData.ims") # an example image
—
Reply to this email directly, view it on GitHub<#149 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AIPFK7ADE7N42GF7AXYFMZDZMWB53AVCNFSM6AAAAABJ4PT3HKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMZRHAYDSNZVHA>.
You are receiving this because you were mentioned.Message ID: ***@***.***>
|
|
@MarikeReimer it sounds like you were able to fix this on your side. I'll close this for now. Feel free to message this thread of this remains an issues for others. |
|
This was only going to be an issue for users with volumetric data, but turned into a non-issues. IMO, it would be best for everyone with that use case to convene in Blender. Also, writing NWB files works better as a button on the NeuroMorphovis Blender addon than as a standalone. @marwan-abdellah Do you have some time to chat about this? |
|
@MarikeReimer Indeed. But I need to get a bit of a context. |
|
At an NWB hackathon a few years back, @bendichter had an insight for storing morphology metadata. We recently used it for a Blender-based dendritic spine study: https://pmc.ncbi.nlm.nih.gov/articles/PMC11573899/. His method is broadly useful; adding that functionality to Neuro/VessMorphoVis, would enable exporting via NWB/saving on DANDI Archives. I used his method to make some example outputs of segmented neurons: This one used an Allen Institute model with NeuroMorphoVis and shows how curve/spline data could be structured: The other combines the AI model with one from the MICrONS/NEURD workflows and shows how dendritic spine morphology data could be structured in a SONOTA framework: https://dandiarchive.org/dandiset/001248/draft/files?location=sub-FrankenNeuron-ses-Demo&page=1 A collaboration feels like it could be a big win for everyone. I know I'm not the only one who wants a tool that does this. :) |
Bug description
I was following the pynwb tutorial to prepare my image data for DANDI Archives:
https://pynwb.readthedocs.io/en/stable/tutorials/domain/images.html#sphx-glr-tutorials-domain-images-py
The Pillow library in their code took issue with our .ims format image data. A workaround with minimal coding exists and it would be nice to share/document it so that troubleshooting is easier for others.
The relevant import statement:
from PIL import Image
The problematic line of code:
img = Image.open("docs/source/figures/logo_pynwb.png") # an example image
Expected behaviour
When executed, this line should return my .ims format image data in a useable form
img = Image.open("MyData.ims") # an example image
Actual behaviour
Image Image.open() believes that my ims file is an HDF5 file
How to reproduce
ReproduceErrors.txt is a python script based on the tutorial code that I used for troubleshooting. I can provide an .ims file if needed.
Your personal set up
To reproduce fully, you would need to download the Blender-addon code which created my dandiset. I can share a Zenodo link via email to anyone who is interested.
-Development Environment: Visual Studio Code
The workaround is to use ImageJ to export problematic file types as a sequence of .png files and add them to the appropriate image container with a For loop. That is a stumbling block for new users.
These error messages in particular are unfriendly for new users who usually don't know they can inspect variables while troubleshooting and who may otherwise spend time troubleshooting the OSError:
File "C:\Users\meowm\AppData\Roaming\Python\Python310\site-packages\PIL\Image.py", line 941, in convert
self.load()
File "C:\Users\meowm\AppData\Roaming\Python\Python310\site-packages\PIL\ImageFile.py", line 366, in load
raise OSError(msg)
OSError: cannot find loader for this HDF5 file
[2024-06-25 15:01:24,767][ERROR]: Uncaught exception
Traceback (most recent call last):
File "C:\Users\meowm\AppData\Roaming\Blender Foundation\Blender\3.6\scripts\addons\TestAddon\ClassDefinitions.py", line 453, in execute
holder = self.AddPanelData()
File "C:\Users\meowm\AppData\Roaming\Blender Foundation\Blender\3.6\scripts\addons\TestAddon\ClassDefinitions.py", line 379, in AddPanelData
data=np.array(img.convert("RGB")),
File "C:\Users\meowm\AppData\Roaming\Python\Python310\site-packages\PIL\Image.py", line 941, in convert
self.load()
File "C:\Users\meowm\AppData\Roaming\Python\Python310\site-packages\PIL\ImageFile.py", line 366, in load
raise OSError(msg)
OSError: cannot find loader for this HDF5 file
Error: Python:
The text was updated successfully, but these errors were encountered: