Caffe for Pseudo-Random Dropout

Our modified Caffe for pseudo-random dropout. This repository at least runs on Ubuntu 14.04, gcc 4.8, OpenCV 2.4.10, CUDA 8.0, and CUDNN 5.

New Features

• Add inner product dropout layer for GPU acceleration.
  • Up to 2.4 times as fast as dropout + inner product. (forward pass, test on GTX 1080)
• New functions in math_functions.cu to optimize matrix multiplication.

Illustration

• The overall process from sequence generating to forward pass of IPD layer is shown as below. Note that this repository does not contain the part in dashed line box. For code or files of that part, please see Pseudo-Random-Dropout for detail.

Installation

• You may add CXXFLAGS += -std=c++11 in Makefile.Config before compiling.
• For detailed installation instructions of Caffe please search on the internet. For convenience (not recommended) you just need one command make all -j8 (if the dependencies are satisfied).

Training and Testing

• The prototxt for dropout+ip and ipd are added in this repository.
• Logs of the time test on the them can be found in the root (skipping period T ranging 1~64 for ipd).

Example

• The example of usage of inner product dropout layer is as follows:
  • seq_addr is the address of the skipping period sequence file. The file is formatted as in seq_in1024.txt. The first number denotes the length of the sequence, and the rest numbers are elements.
  • Other parameters are the same with that of inner product layer.

layer {
  name: "ipd1"
  type: "InnerProductDropout"
  bottom: "pool3"
  top: "ipd1"
  param {
    lr_mult: 1
  }
  param {
    lr_mult: 2
  }
  inner_product_dropout_param {
    seq_addr: "/home/user/caffe_pr/seq_in1024.txt"
    num_output: 64
    weight_filler {
      type: "gaussian"
      std: 0.1
    }
    bias_filler {
      type: "constant"
    }
  }
}

Caffe

Caffe is a deep learning framework made with expression, speed, and modularity in mind. It is developed by Berkeley AI Research (BAIR)/The Berkeley Vision and Learning Center (BVLC) and community contributors.

Check out the project site for all the details like

• DIY Deep Learning for Vision with Caffe
• Tutorial Documentation
• BAIR reference models and the community model zoo
• Installation instructions

and step-by-step examples.

Custom distributions

• Intel Caffe (Optimized for CPU and support for multi-node), in particular Xeon processors (HSW, BDW, SKX, Xeon Phi).
• OpenCL Caffe e.g. for AMD or Intel devices.
• Windows Caffe

Community

Please join the caffe-users group or gitter chat to ask questions and talk about methods and models. Framework development discussions and thorough bug reports are collected on Issues.

Happy brewing!

License and Citation

Caffe is released under the BSD 2-Clause license. The BAIR/BVLC reference models are released for unrestricted use.

Please cite Caffe in your publications if it helps your research:

@article{jia2014caffe,
  Author = {Jia, Yangqing and Shelhamer, Evan and Donahue, Jeff and Karayev, Sergey and Long, Jonathan and Girshick, Ross and Guadarrama, Sergio and Darrell, Trevor},
  Journal = {arXiv preprint arXiv:1408.5093},
  Title = {Caffe: Convolutional Architecture for Fast Feature Embedding},
  Year = {2014}
}