About two weeks ago I was writing a application doing some camera motion estimation and I wanted to use the SFM OpenCV module to resolve the camera parameters. Now you can imagine what have been my surprise when I found out that not only the SFM module does not come with default OpenCV build but you need some dirty hack to make the Python bindings available.
I have then decided to write that down in case it can be of some benefit for someone else. Without further ado let's dive in the compiling pipeline of OpenCV (actually not very much).
Installing dependencies
The first step to compile OpenCV is installing its dependencies, this is identical from a standard OpenCV build.
# apt install build-essential
# apt install cmake git libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev
# apt install python-dev python-numpy libtbb2 libtbb-dev libjpeg-dev libpng-dev libtiff-dev libjasper-dev libdc1394-22-dev
This install the build dependencies and some optional libraries for modules. Then comes the dependencies of the SFM module, once again nothing too special
# apt install libeigen3-dev libgflags-dev libgoogle-glog-dev libatlas-base-dev libsuitesparse-dev
The only thing is that one library is not in the packages repository and have to be compiled, so here we go. First let's create a directory for all the cloned repos to be put on.
$ mkdir ~/opencv_build
$ cd ~/opencv_build
And then clone and install Ceres. By the way, if you wonder, this lib is used to solve the non linear quadratic optimisation problem involve in find the camera parameters.
$ git clone https://ceres-solver.googlesource.com/ceres-solver
$ cd ceres-solver
$ mkdir build && cd build
$ make -j4
$ sudo make install
If you have a CPU with more than four threads then change the 4 in the make -j4 above accordingly (just so you can use your full power).
Now we are supposed to be all good, if not cmake will tell you anyway.
Patching SFM
Let's clone the opencv and opencv_contrib module, if you prefer to use a more stable version than the master, you can download a stable version and extract it in your working directory instead.
$ cd ~/opencv_build
$ git clone https://github.com/opencv/opencv.git
$ git clone https://github.com/opencv/opencv_contrib.git
Then edit with your favorite text editor the reconstruct.hpp header file.
vim opencv_contrib/modules/sfm/include/opencv2/sfm/reconstruct.hpp
Now with the upmost care (because I like dramatic effect) change the CV_EXPORTS for something more sensible because right now the bindings are not being exported.
If you are gonna use only one of the overloaded reconstruct you could safely replace the one of your needs by CV_EXPORTS_W but if, like me, you want several bindings to be available then you need to rename the function using CV_EXPORTS_AS. That being said you will loose compatibility with other version of OpenCV depending of the way it has been compiled.
So in my case what I have done is something like this.
// Fist declaration
CV_EXPORTS_AS(reconstructProj)
void
reconstruct(...);
// Second declaration
CV_EXPORTS_AS(reconstructRotTrans)
void
reconstruct(...);
// Nothing changed for the third and fourth.
Note that I am exporting only the first two declarations as it is what I need but you could totally export the remaining two. Also the name given to CV_EXPORTS_AS is the name that we will be using in Python, the C++ usage remains the same.
The functions parameters also need now to be changed. If untouched we will run in some dimension problems as some output arrays (Ps, Rs, Ts, points3d) are actually arrays of arrays, the three first are arrays of transformation matrices and the fourth is an array of vectors.
Finally I have removed one of the ifdef precondition statements to force the declaration of the functions, not seen at the Python binding compilation stage otherwise. In the end my final reconstruct.hpp file looks like that (without documentation comments). Note the OutputArrayOfArrays in the declaration!
Again you can apply the change to the last two declaration to fit your needs.
#ifndef __OPENCV_SFM_RECONSTRUCT_HPP__
#define __OPENCV_SFM_RECONSTRUCT_HPP__
#include <vector>
#include <string>
#include <opencv2/core.hpp>
namespace cv
{
namespace sfm
{
//! @addtogroup reconstruction
//! @{
CV_EXPORTS_AS(reconstructProj)
void
reconstruct(InputArrayOfArrays points2d, OutputArrayOfArrays Ps, OutputArrayOfArrays points3d,
InputOutputArray K, bool is_projective = false);
CV_EXPORTS_AS(reconstructRotTrans)
void
reconstruct(InputArrayOfArrays points2d, OutputArrayOfArrays Rs, OutputArrayOfArrays Ts,
InputOutputArray K, OutputArrayOfArrays points3d, bool is_projective = false);
CV_EXPORTS
void
reconstruct(const std::vector<String> images, OutputArray Ps, OutputArray points3d,
InputOutputArray K, bool is_projective = false);
CV_EXPORTS
void
reconstruct(const std::vector<String> images, OutputArray Rs, OutputArray Ts,
InputOutputArray K, OutputArray points3d, bool is_projective = false);
//! @} sfm
} /* namespace cv */
} /* namespace sfm */
#endif
Compiling OpenCV
Then we can resume our compilation work, change directory to the opencv directory and compile it.
$ cd ~/opencv
$ mkdir build
$ cd build
$ cmake -D CMAKE_BUILD_TYPE=Release -D CMAKE_INSTALL_PREFIX=/usr/local -D OPENCV_EXTRA_MODULES_PATH=~/opencv_build/opencv_contrib/modules -D BUILD_SHARED_LIBS=ON -D BUILD_opencv_sfm=ON -D OPENCV_ENABLE_NONFREE=ON -D BUILD_SHARED_LIBS=ON -D BUILD_EXAMPLES=OFF -D BUILD_TESTS=OFF -D OPENCV_GENERATE_PKGCONFIG=ON ..
$ make -j4
$ sudo make install
If no error happened during the build steps (and hopefully it is the case) you should be able to access the sfm module within Python!
$ python
Python 3.8.1 (default, Jan 22 2020, 06:38:00)
[GCC 9.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import cv2
>>> cv2.sfm.reconstructProj
<built-in function reconstructProj>