Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Should object arrays be allowed #47

Open
simetenn opened this issue May 11, 2018 · 3 comments
Open

Should object arrays be allowed #47

simetenn opened this issue May 11, 2018 · 3 comments

Comments

@simetenn
Copy link
Contributor

We should decide if we should allow object arrays or not. There are several advantages and disadvantages to this. Currently we allow object arrays, but we have not made a conscious decision to support them. They just happen to work.

@dragly
Copy link
Member

dragly commented Nov 8, 2018

They should not be allowed and no longer seem to work because memmap fails in this case. I think we should catch this before memmap does and just check dtype.hasobject. If true, we raise an exception saying that this is not supported.

@espenhgn
Copy link

espenhgn commented Nov 8, 2018

Also not supported w. h5py:

In [8]: f = h5py.File('objecttest.h5', 'w')

In [9]: f['o'] = np.array([object()])
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-9-ff5a86282ebb> in <module>()
----> 1 f['o'] = np.array([object()])

h5py/_objects.pyx in h5py._objects.with_phil.wrapper()

h5py/_objects.pyx in h5py._objects.with_phil.wrapper()

/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/h5py/_hl/group.pyc in __setitem__(self, name, obj)
    289 
    290         else:
--> 291             ds = self.create_dataset(None, data=obj, dtype=base.guess_dtype(obj))
    292             h5o.link(ds.id, self.id, name, lcpl=lcpl)
    293 

/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/h5py/_hl/group.pyc in create_dataset(self, name, shape, dtype, data, **kwds)
    106         """
    107         with phil:
--> 108             dsid = dataset.make_new_dset(self, shape, dtype, data, **kwds)
    109             dset = dataset.Dataset(dsid)
    110             if name is not None:

/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/h5py/_hl/dataset.pyc in make_new_dset(parent, shape, dtype, data, chunks, compression, shuffle, fletcher32, maxshape, compression_opts, fillvalue, scaleoffset, track_times)
     95         else:
     96             dtype = numpy.dtype(dtype)
---> 97         tid = h5t.py_create(dtype, logical=1)
     98 
     99     # Legacy

h5py/h5t.pyx in h5py.h5t.py_create()

h5py/h5t.pyx in h5py.h5t.py_create()

h5py/h5t.pyx in h5py.h5t.py_create()

TypeError: Object dtype dtype('O') has no native HDF5 equivalent

@dragly
Copy link
Member

dragly commented Nov 9, 2018

Good point. I think we should also raise a TypeError to have as close to the same behavior as h5py as possible. That eases porting from h5py to Exdir.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants