How I ran torch-hdf5 on Windows (valid HDF5_INCLUDE_PATH and HDF5_LIBRARIES)

[saippuakauppias]

Will tell you my little (but painful) experience run torch-hdf5 on Windows.
First I will mention that it worked on Win7 x64 SP1 and Win10 x64.
And that I tried to run torch-rnn ( https://github.com/jcjohnson/torch-rnn ) with distro-win ( https://github.com/BTNC/distro-win ).

After a lot of fixes for distro-win on Win7 (surprisingly, Win10, it booted almost immediately) I started to install torch-hdf5 approximately the same as written here: https://github.com/crisbal/docker-torch-rnn/blob/master/Base/Dockerfile#L42-L44 (given the fact that I am now in windows and I have enabled the environment in Anaconda + has already been launched torch-activate.bat + I already installed hdf5 in conda in my Anaconda env).

(torch-vc14) C:\distro-win\pkg> git clone https://github.com/deepmind/torch-hdf5
(torch-vc14) C:\distro-win\pkg> cd torch-hdf5
(torch-vc14) C:\distro-win\pkg\torch-hdf5> luarocks make hdf5-0-0.rockspec
... "bla-bla-bla" ... ERROR!

I do not remember what was the error but it was due to the following reasons:
In the build command in https://github.com/deepmind/torch-hdf5/blob/master/hdf5-0-0.rockspec#L21-L23 contains ";", after remove it and run same commands with hands:

(torch-vc14) C:\distro-win\pkg\torch-hdf5> cmake -E make_directory build
(torch-vc14) C:\distro-win\pkg\torch-hdf5> cd build
(torch-vc14) C:\distro-win\pkg\torch-hdf5> cmake .. -DCMAKE_BUILD_TYPE=Release -DCMAKE_PREFIX_PATH="$(LUA_BINDIR)/.." -DCMAKE_INSTALL_PREFIX="$(PREFIX)"

^^^^^ replace $(LUA_BINDIR)/.. and $(PREFIX) those that appear above you in the time of the error

(torch-vc14) C:\distro-win\pkg\torch-hdf5> nmake install

^^^^^ I remove cd build because I already in this directory(!). And nmake - this replaced by my command from bad output.

After this instructions torch-hdf5 installed, but sometimes in the wrong directory, you need (I don't know why, probably because some paths). If you run the command th -e "require 'hdf5'" you'll see an error that the package of torch-hdf5 is not found, then you need to copy all files from: C:\distro-win\install\luarocks\systree\lib\luarocks\rocks\hdf5\0-0\lua to C:\distro-win\install\luarocks\systree\share\lua\5.1\hdf5

Then running the command th -e "require 'hdf5'" will output something like this:

Error: unable to find a valid HDF5 lib path in the config

To fix this you need to change HDF5_LIBRARIES in the file C:\distro-win\install\luarocks\systree\share\lua\5.1\hdf5\config.lua to the following value:

C:/Anaconda2/envs/torch-vc14/Library/bin/hdf5.dll

(But it is worth considering that there is spelled out the path to an installed hdf5 in MY Anaconda environment)

And (very important!) to change the file C:\distro-win\install\luarocks\systree\share\lua\5.1\hdf5\ffi.lua ( https://github.com/deepmind/torch-hdf5/blob/41d87d827c431041cb679d7f39ad7b2868516bc9/luasrc/ffi.lua#L13 ) from

if name == 'libhdf5' then

to:

if name == 'hdf5' then

Because otherwise there are usually substituted libhdf5.lib, but when I try to run local hdf5lib = ffi.load(hdf5LibPath) with this file you will see: %1 эх ты x∈ё Win32

This means that you need to pass in ffi.load the name of library with extension *.dll on Win (!!!!!!!)

After this hotfix, when you run th -e "require 'hdf5'" you most likely will see the error:

Error: unable to locate HDF5 header file at

This is due to the fact that you have prescribed is not the right path in HDF5_INCLUDE_PATH. I corrected him on the path:

C:/Anaconda2/envs/torch-vc14/Library/include

(But it is worth considering that there is spelled out the path to an installed hdf5 in MY Anaconda environment)

Then when you start th -e "require 'hdf5'" you can get around this error:

gcc was not found.

To correct this you need to:

1. Install MinGW
2. To set the Path environment variable (in Windows) with the following: ...;C:\MinGW\bin <- note that here the path to an installed MinGW in MY SYSTEM
3. Restart "VS* * Tools Command Prompt"
4. To re-activate the Anaconda environment and run torch-activate.bat

Then when you run the command th -e "require 'hdf5'" you will NOT see errors.

In conclusion, post below how looks the file config.lua I have:

hdf5._config = {
HDF5_INCLUDE_PATH = "C:/Anaconda2/envs/torch-vc14/Library/include",
HDF5_LIBRARIES = "C:/Anaconda2/envs/torch-vc14/Library/bin/hdf5.dll"
}

Note that you paths may be others.

I hope that the developers will fix this package under Win and no one will get the same pain and suffering as I am.

[akfidjeland]

Project owners will not be adding fixes for Windows, but patches will be accepted.

[saippuakauppias]

I do not write in Lua, so sort it out yourself :)