Mac OSX (EI Capitan)搭建Caffe环境并配置python接口

天成1

　　Caffe是一个清晰而高效的深度学习框架，其作者是博士毕业于UC Berkeley的贾扬清。Caffe是纯粹的C++/CUDA架构，支持命令行、Python和MATLAB接口；可以在CPU和GPU直接无缝切换。我在MacbookPro（无NVIDIA显卡）上大费周章地配置了Caffe的环境，并花了许多时间配置其python接口。


一、下载Caffe

github上的下载地址：https://github.com/BVLC/caffe
进入到下载后的路径，并复制 Makefile.config.example 重命名为 Makefile.config (我电脑的用户名是cuiqi，注意修改)


git clone https://github.com/BVLC/caffe.git
cd /Users/cuiqi/Downloads/caffe-master && cp Makefile.config.example Makefile.config
二、安装相关依赖
　　对于需要python接口的情况，需要以下依赖



1. CUDA
由于我的Mac没有NVIDIA的GPU，所以只能使用CPU_ONLY模式，需要在 Makefile.config 中修改
CPU_ONLY := 1

2. BLAS via ATLAS, MKL, or OpenBLAS.
# Basic Linear Algebra Subprograms，基础线性代数程序集

3. Boost >= 1.55
# Deepdream是用Python接Caffe，因此还需要 boost.python 支持


brew install boost --with-python
brew install boost-python

4. OpenCV >= 2.4 including 3.0

5. protobuf, glog, gflags


brew install protobuf
brew install glog
brew install gflags
6. IO libraries hdf5, leveldb, snappy, lmdb



brew install leveldb
brew install lmdb

brew tap homebrew/science
brew install homebrew/science/hdf5
# python driver for hdf5


pip install h5py
7. numpy for python


brew install numpy
三、修改Makefile.config中相应的路径
　　如果要使用Anaconda的python环境可以在Makefile.config中取消相应的注释，我试过这样做，可是在CMAKE的时候并不奏效：



Python:
Interpreter : /usr/bin/python2.7 (ver. 2.7.10)
Libraries : /usr/lib/libpython2.7.dylib (ver 2.7.10)
NumPy : /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/numpy/core/include (ver 1.8.0rc1)
　　所以我放弃了使用Anaconda，改使用系统自带的python。



PYTHON_INCLUDE := /usr/include/python2.7  
　　由于我使用了Homebrew安装了numpy，所以我在makefile.config中修改相应的numpy路径



PYTHON_INCLUDE := /usr/include/python2.7 \
/usr/local/Cellar/numpy/1.11.2/lib/python2.7/site-packages/numpy/core/include/numpy/core/include
　　我的Makefile.config文件：





## Refer to http://caffe.berkeleyvision.org/installation.html
# Contributions simplifying and improving our build system are welcome!
# cuDNN acceleration switch (uncomment to build with cuDNN).
# USE_CUDNN := 1
# CPU-only switch (uncomment to build without GPU support).
CPU_ONLY := 1
# uncomment to disable IO dependencies and corresponding data layers
# USE_OPENCV := 0
# USE_LEVELDB := 0
# USE_LMDB := 0
# uncomment to allow MDB_NOLOCK when reading LMDB files (only if necessary)
#    You should not set this flag if you will be reading LMDBs with any
#    possibility of simultaneous read and write
# ALLOW_LMDB_NOLOCK := 1
# Uncomment if you're using OpenCV 3
# OPENCV_VERSION := 3
# To customize your choice of compiler, uncomment and set the following.
# N.B. the default for Linux is g++ and the default for OSX is clang++
# CUSTOM_CXX := g++
# CUDA directory contains bin/ and lib/ directories that we need.
CUDA_DIR := /usr/local/cuda
# On Ubuntu 14.04, if cuda tools are installed via
# "sudo apt-get install nvidia-cuda-toolkit" then use this instead:
# CUDA_DIR := /usr
# CUDA architecture setting: going with all of them.
# For CUDA < 6.0, comment the *_50 lines for compatibility.
CUDA_ARCH := -gencode arch=compute_20,code=sm_20 \
-gencode arch=compute_20,code=sm_21 \
-gencode arch=compute_30,code=sm_30 \
-gencode arch=compute_35,code=sm_35 \
-gencode arch=compute_50,code=sm_50 \
-gencode arch=compute_50,code=compute_50
# BLAS choice:
# atlas for ATLAS (default)
# mkl for MKL
# open for OpenBlas
BLAS := atlas
# Custom (MKL/ATLAS/OpenBLAS) include and lib directories.
# Leave commented to accept the defaults for your choice of BLAS
# (which should work)!
# BLAS_INCLUDE := /path/to/your/blas
# BLAS_LIB := /path/to/your/blas
# Homebrew puts openblas in a directory that is not on the standard search path
# BLAS_INCLUDE := $(shell brew --prefix openblas)/include
# BLAS_LIB := $(shell brew --prefix openblas)/lib
# This is required only if you will compile the matlab interface.
# MATLAB directory should contain the mex binary in /bin.
# MATLAB_DIR := /usr/local
# MATLAB_DIR := /Applications/MATLAB_R2012b.app
# NOTE: this is required only if you will compile the python interface.
# We need to be able to find Python.h and numpy/arrayobject.h.
PYTHON_INCLUDE := /usr/include/python2.7 \
/usr/local/Cellar/numpy/1.11.2/lib/python2.7/site-packages/numpy/core/include/numpy/core/include
# Anaconda Python distribution is quite popular. Include path:
# Verify anaconda location, sometimes it's in root.
# ANACONDA_HOME := $(HOME)/anaconda
# PYTHON_INCLUDE := $(ANACONDA_HOME)/include \
# $(ANACONDA_HOME)/include/python2.7 \
# $(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include \
# Uncomment to use Python 3 (default is Python 2)
# PYTHON_LIBRARIES := boost_python3 python3.5m
# PYTHON_INCLUDE := /usr/include/python3.5m \
#                 /usr/lib/python3.5/dist-packages/numpy/core/include
# We need to be able to find libpythonX.X.so or .dylib.
PYTHON_LIB := /usr/lib
# PYTHON_LIB := $(ANACONDA_HOME)/lib
# Homebrew installs numpy in a non standard path (keg only)
# PYTHON_INCLUDE += $(dir $(shell python -c 'import numpy.core; print(numpy.core.__file__)'))/include
# PYTHON_LIB += $(shell brew --prefix numpy)/lib
# Uncomment to support layers written in Python (will link against Python libs)
# WITH_PYTHON_LAYER := 1
# Whatever else you find you need goes here.
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib
# If Homebrew is installed at a non standard location (for example your home directory) and you use it for general dependencies
# INCLUDE_DIRS += $(shell brew --prefix)/include
# LIBRARY_DIRS += $(shell brew --prefix)/lib
# Uncomment to use `pkg-config` to specify OpenCV library paths.
# (Usually not necessary -- OpenCV libraries are normally installed in one of the above $LIBRARY_DIRS.)
# USE_PKG_CONFIG := 1
# N.B. both build and distribute dirs are cleared on `make clean`
BUILD_DIR := build
DISTRIBUTE_DIR := distribute
# Uncomment for debugging. Does not work on OSX due to https://github.com/BVLC/caffe/issues/171
# DEBUG := 1
# The ID of the GPU that 'make runtest' will use to run unit tests.
TEST_GPUID := 0
# enable pretty build (comment to see full commands)
Q ?= @
View Code
四、编译
　　使用cmake工具，将cmake的sourcecode设为caffe的路径，在caffe目录下建立build文件夹，并将cmake的build路径设为该build的路径，configure，generate。
　　此时打开终端工具进入build目录下，make。



make all
make test
make runtest
make pycaffe
　　如果没有出现错误，并全部通过后，此时make告一段落，但是并不能确定可以在python中import。
　　在终端上：



vi etc/profile
　　或



sudo vi /etc/profile
　　进入修改系统的环境变量，将 caffe/python/添加到Python系统路径：

export PYTHONPATH=path to caffe/python:$PYTHONPATH
例如：



export PYTHONPATH=/Users/cuiqi/Downloads/caffe-master/python:$PYTHONPATH
　　终端输入python，确认这个python环境是刚刚在makefile.config中设定的那个，可以 which python 确认。由于我没有使用Anaconda，我的是
　　/usr/local/bin/python ，确认为刚刚在makefile.config中设定的路径。
　　>>> import caffe
　　# 可能的错误
　　　　ImportError: No module named skimage.io
　　  # 解决
　　   pip install scikit-image
　　参考：https://github.com/rainyear/lolita/issues/10?utm_source=tuicool&utm_medium=referral

　　16.12.22更新：
　　还可能出现的错误：segment fault：11
　　在 ITNOSE 上一作者认为此错误很有可能是系统里也有多个版本的Python，并且编译和运行的时候不是一个版本。
　　解决办法：
　　当然就是要统一编译与运行的python版本。终端键入 which python ，反馈给我/usr/local/bin/python，而CMAKE工具反馈给我的是系统自带的python路径。
　　1.对于我而言，我准备使用系统自带的python。使用Homebrew工具brew install python后，python环境路径为/usr/local/bin/python，而caffe编译时使用了系统自带的python路径     /System/Library/Frameworks/Python.framework/Versions/2.7/bin/python。这个路径可以使用otool反编译工具检出，例如：otool -L python/caffe/_caffe.so （_caffe.so在下载好的caffe的python/caffe下）。
　　2.修改bash_profile配置文件以在启动系统时使用自带python，打开终端，键入 vi ~/.bash_profile ，输入：



# Setting PATH for Python 2.7
# The orginal version is saved in .bash_profile.pysave
PATH="/System/Library/Frameworks/Python.framework/Versions/2.7/bin:${PATH}"
export PATH
　　完成后输入：wq（保存退出），终端键入 source ~/.bash_profile 或者注销及重启使修改生效。
　　3.再次终端键入 which python，返回 /System/Library/Frameworks/Python.framework/Versions/2.7/bin/python，说明修改生效。键入python，
　　>>> import caffe
　　无报错，成功。
　　参考：
　　http://www.itnose.net/detail/6522348.html
　　https://www.zhihu.com/question/30941329

　　16.12.29更新：
　　可能出现的错误：TypeError: __init__() got an unexpected keyword argument 'syntax'；
　　参考dupuleng的解决方法，这是由于protobuf版本的问题造成的。
　　查看protobuf版本：



pip show protobuf
　　protobuf信息：



Name: protobuf
Version: 2.6.1
Summary: Protocol Buffers
Home-page: https://developers.google.com/protocol-buffers/
Author: protobuf@googlegroups.com
Author-email: protobuf@googlegroups.com
License: New BSD License
Location: /Users/cuiqi/anaconda/lib/python2.7/site-packages
Requires: setuptools
　　此版本（2.6.1）需要变更：



pip uninstall protobuf
pip install 'protobuf>=3.0.0a3'


pip show numpy protobuf
　　protobuf信息：



Name: protobuf
Version: 3.1.0.post1
Summary: Protocol Buffers
Home-page: https://developers.google.com/protocol-buffers/
Author: protobuf@googlegroups.com
Author-email: protobuf@googlegroups.com
License: New BSD License
Location: /Users/cuiqi/anaconda/lib/python2.7/site-packages
Requires: six, setuptools
　　完成


关于cmake：
　　经过我的测试发现修改makefile.config后，cmake的 configure之前还要手动修改python的相关路径，否则cmake会使用mac的系统python环境，造成上述segment fault 11的情况。关键在于要勾选Advanced选项：

　　不勾选Advanced：

　　更改好之后再configure，generate，cmake就会选择相应的环境路径了。