Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HDF5 unexpectedly fails #71

Open
rasoolfa opened this issue Jul 20, 2016 · 0 comments
Open

HDF5 unexpectedly fails #71

rasoolfa opened this issue Jul 20, 2016 · 0 comments

Comments

@rasoolfa
Copy link

When I run the following simple code, the hdf5 throws some errors [when i> 5600].

require 'torch'
require 'hdf5'
local chunked_h5_file = hdf5.open('test.h5', 'w')
local ii_data = torch.Tensor()
local dtype= 'torch.FloatTensor'
for i = 1,  10000 do
    chunked_h5_file:write('vi'..tostring(i),ii_data:resize(10,10):zero():type(dtype))  
end

Here is the error message:

HDF5-DIAG: Error detected in HDF5 (1.8.11) thread 140328551929728:
  #000: ../../../src/H5D.c line 170 in H5Dcreate2(): unable to create dataset
    major: Dataset
    minor: Unable to initialize object
  #001: ../../../src/H5Dint.c line 439 in H5D__create_named(): unable to create and link to dataset
    major: Dataset
    minor: Unable to initialize object
  #002: ../../../src/H5L.c line 1638 in H5L_link_object(): unable to create new link to object
    major: Links
    minor: Unable to initialize object
  #003: ../../../src/H5L.c line 1882 in H5L_create_real(): can't insert link
    major: Symbol table
    minor: Unable to insert object
  #004: ../../../src/H5Gtraverse.c line 861 in H5G_traverse(): internal path traversal failed
    major: Symbol table
    minor: Object not found
  #005: ../../../src/H5Gtraverse.c line 641 in H5G_traverse_real(): traversal operator failed
    major: Symbol table
    minor: Callback failed
  #006: ../../../src/H5L.c line 1685 in H5L_link_cb(): unable to create object
    major: Object header
    minor: Unable to initialize object
  #007: ../../../src/H5O.c line 3015 in H5O_obj_create(): unable to open object
    major: Object header
    minor: Can't open object
  #008: ../../../src/H5Doh.c line 293 in H5O__dset_create(): unable to create dataset
    major: Dataset
    minor: Unable to initialize object
  #009: ../../../src/H5Dint.c line 1048 in H5D__create(): can't update the metadata cache
    major: Dataset
    minor: Unable to initialize object
  #010: ../../../src/H5Dint.c line 802 in H5D__update_oh_info(): unable to create dataset object header
    major: Dataset
    minor: Unable to initialize object
  #011: ../../../src/H5O.c line 1257 in H5O_create(): unable to cache object header
    major: Object header
    minor: Unable to insert object
  #012: ../../../src/H5AC.c line 986 in H5AC_insert_entry(): H5C_insert_entry() failed
    major: Object cache
    minor: Unable to insert metadata into cache
  #013: ../../../src/H5C.c line 2699 in H5C_insert_entry(): H5C_make_space_in_cache failed.
    major: Object cache
    minor: Unable to insert metadata into cache
  #014: ../../../src/H5C.c line 8223 in H5C_make_space_in_cache(): unable to flush entry
    major: Object cache
    minor: Unable to flush data from cache
  #015: ../../../src/H5C.c line 7749 in H5C_flush_single_entry(): unable to flush entry
    major: Object cache
    minor: Unable to flush data from cache
  #016: ../../../src/H5Ocache.c line 487 in H5O_flush(): unable to write object header chunk to disk
    major: Object header
    minor: Write failed
  #017: ../../../src/H5Fio.c line 158 in H5F_block_write(): write through metadata accumulator failed
    major: Low-level I/O
    minor: Write failed
  #018: ../../../src/H5Faccum.c line 640 in H5F_accum_write(): file write failed
    major: Low-level I/O
    minor: Write failed
  #019: ../../../src/H5FDint.c line 185 in H5FD_write(): driver write request failed
    major: Virtual File Layer
    minor: Write failed
  #020: ../../../src/H5FDsec2.c line 822 in H5FD_sec2_write(): file write failed: time = Wed Jul 20 05:29:00 2016
, filename = '/hdfs/test.h5', file descriptor = 3, errno = 5, error message = 'Input/output error', buf = 0x44211a8, total write size = 144, bytes this sub-write = 144, bytes actually written = 18446744073709551615, offset = 274432
    major: Low-level I/O
    minor: Write failed
HDF5-DIAG: Error detected in HDF5 (1.8.11) thread 140328551929728:
  #000: ../../../src/H5Dio.c line 234 in H5Dwrite(): can't prepare for writing data
    major: Dataset
    minor: Write failed
  #001: ../../../src/H5Dio.c line 266 in H5D__pre_write(): not a dataset
    major: Invalid arguments to routine
    minor: Inappropriate type
../pkg/torch/install/bin/luajit: ../pkg/torch/install/share/lua/5.1/hdf5/group.lua:115: Error writing data vi5686 to [HDF5Group 33554432 /]
stack traceback:
    [C]: in function 'error'
    .../pkg/torch/install/share/lua/5.1/hdf5/group.lua:115: in function '_writeData'
    .../pkg/torch/install/share/lua/5.1/hdf5/group.lua:292: in function '_write_or_append'
    .../pkg/torch/install/share/lua/5.1/hdf5/group.lua:255: in function </pkg/torch/install/share/lua/5.1/hdf5/group.lua:254>
    ...g/pkg/torch/install/share/lua/5.1/hdf5/file.lua:69: in function '_write_or_append'
    ...g/pkg/torch/install/share/lua/5.1/hdf5/file.lua:43: in function 'write'
    check_hd5.lua:9: in main chunk
    [C]: in function 'dofile'
    .../pkg/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:145: in main chunk
    [C]: at 0x00406670
Segmentation fault (core dumped)

I have run this codes on different machines, it fails only on one machine. I installed fresh Ubuntu 14.04. Don't know what is the problem. I tried to update different libraries and packages but no luck.

Any help would be appreciated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant