Missing shared object file when run caffe-classification

[yzh89]

I followed the tutorial and finished installation of CK-caffe. compiling of classification has no errors, but when running, the system complain about missing .so file. The error file is attached below

export CK_DATASET_FILENAME=data.jpg
export CK_CAFFE_MODEL_WEIGHTS=/home/yzhou/CK_TOOLS/caffemodel-bvlc-googlenet-fast-mirror/bvlc_googlenet.caffemodel
export CK_CAFFE_BATCH_SIZE=1
export CK_ENV_MODEL_CAFFE_WEIGHTS=/home/yzhou/CK_TOOLS/caffemodel-bvlc-googlenet-fast-mirror/bvlc_googlenet.caffemodel
export CK_CAFFE_MODEL_MEAN_BIN=imagenet_mean.binaryproto

export CK_CAFFE_ITERATIONS=1
export CK_CAFFE_MODEL=/home/yzhou/CK_REPOS/ck-caffe/program/caffe-classification/tmp/tmp-DMbpcR.prototxt
export CK_CAFFE_MODEL_FILE=tmp-DMbpcR.prototxt


echo    executing code ...
 ./classification ${CK_CAFFE_MODEL_FILE} ${CK_CAFFE_MODEL_WEIGHTS} ../imagenet_mean.binaryproto ../synset_words.txt /home/yzhou/CK_REPOS/ctuning-datasets-min/dataset/image-jpeg-0001/data.jpg > tmp-output1.tmp 2> tmp-output2.tmp

***************************************************************************************
  (bash -c "chmod 755 ./tmp-agReEJ.sh; . ./tmp-agReEJ.sh")

  (sleep 0.5 sec ...)

  (run ...)
executing code ...

 (printing output files) 

    * tmp-output1.tmp

      

    * tmp-output2.tmp

      ./classification: error while loading shared libraries: libcaffe.so.1.0.0: cannot open shared object file: No such file or directory
      

  (post processing from script  /  ... )"


  (reading fine grain timers from tmp-ck-timer.json ...)


Program execution likely failed (can't find fine grain timers)!

[psyhtest]

yzh89 Thanks to our offline investigation, it is now clear what's going wrong.

CK sets up the path to the Caffe library as follows:

export CK_ENV_LIB_CAFFE_LIB=$CK_TOOLS/lib-caffe-bvlc-master-cpu-master-gcc-4.8.4-linux-64/install/lib

However, on your Ubuntu 14.04 platform the Caffe library gets installed under $CK_TOOLS/lib-caffe-bvlc-master-cpu-master-gcc-4.8.4-linux-64/install/lib/x86_64-linux, i.e. one level down in the x86_64-linux subdirectory.

This actually appears to be quite characteristic of your platform: even most system libraries are under /usr/lib/x86_64-linux-gnu, not under /usr/lib as is usual on other platforms. For example, here's a snippet from ldd $CK_REPOS/ck-caffe/program/caffe-classification/tmp/classification:

    linux-vdso.so.1 =>  (0x00007ffc9b5d5000)
    libcaffe.so.1.0.0 => /home/yzhou/CK_TOOLS/lib-caffe-bvlc-master-cpu-master-gcc-4.8.4-linux-64/install/lib/libcaffe.so.1.0.0 (0x00007f439c845000)
    libglog.so.0 => /usr/lib/x86_64-linux-gnu/libglog.so.0 (0x00007f439c60c000)
    libopencv_core.so.2.4 => /usr/lib/x86_64-linux-gnu/libopencv_core.so.2.4 (0x00007f439c1d5000)
    librtlxopenme.so => /home/yzhou/CK_TOOLS/lib-rtl-xopenme-0.3-gcc-4.8.4-linux-64/lib/librtlxopenme.so (0x00007f439bfd2000)
    libboost_thread.so.1.54.0 => /usr/lib/x86_64-linux-gnu/libboost_thread.so.1.54.0 (0x00007f439bdbb000)
    libboost_filesystem.so.1.54.0 => /usr/lib/x86_64-linux-gnu/libboost_filesystem.so.1.54.0 (0x00007f439bba5000)
    libboost_system.so.1.54.0 => /usr/lib/x86_64-linux-gnu/libboost_system.so.1.54.0 (0x00007f439b9a1000)
...

As I understand, the following quick workaround has worked for you:

$ cp $CK_TOOLS/lib-caffe-bvlc-master-cpu-master-gcc-4.8.4-linux-64/install/lib/x86_64-linux/* \
        $CK_TOOLS/lib-caffe-bvlc-master-cpu-master-gcc-4.8.4-linux-64/install/lib/

A proper fix is to make CK include correct paths into the environment script. We will look into this.

The next problem you encountered was with the system Boost 1.54.0 being used instead of the CK-installed Boost 1.62.0. This appears to be the same I've dealt with recently: #113. You need to reinstall Caffe e.g.

$ ck clean env --tags=lib,caffe,vcpu
$ ck pull all
$ ck install package:lib-caffe-bvlc-master-cpu-universal

(and apply the same workaround as above).