Data preprocessing

Datalayers

There are several data layers each of which has its strengths and weaknesses.

DataLayer

• Only good for 1-of-k classification
• Use this if possible

ImageDataLayer

• Get images and labels directly from image files
• No LMDB but probably slower than DataLayer

HDF5Layer

• Reads arbitrary data from HDF5 files (easy to read / write in Python using h5py)
• Good for any task - regression, etc
• Other DataLayers do prefetching in a separate thread, HDF5Layer does not
• Can only store float32 and float64 data - no uint8 means --> image data will be huge
• Use this if you have to

MemoryLayer

• Manually copy data into the network
• Slow; don’t use this for training
• Useful for quickly visualizing results

Source: http://vision.stanford.edu/teaching/cs231n/slides/caffe_tutorial.pdf

For mor information on how to import data see Import Datasets