OpenCV2马拉松第10圈——直方图反向投影(back project)

小程序：扫一扫查出行
【扫一扫了解最新限行尾号】
复制小程序

收入囊中

灰度图像的反向投影
彩色图像的反向投影
利用反向投影做object detect

葵花宝典

什么是反向投影？事实上没有那么高大上！

在上一篇博文学到，图像能够获得自己的灰度直方图.

反向投影差点儿相同是逆过程，由直方图得到我们的投影图。

步骤例如以下：

依据模版图像，得到模版图像的灰度直方图.
对灰度直方图对归一化，归一化后是个概率分布，直方图的积分是1
依据概率分布的直方图，求输入图像的投影图，也就是对每个像素点，我们依据灰度值，能够得到其概率
得到的投影图是介于[0,1]之间的，为了方便显示，要作对应的扩大，一般以255最经常使用

举个样例，我们取右边方框作模版图像

我们再来看以下的图：

先看after project,很暗，由于概率直方图的值都很小，要知道平均值1/255是多么小，扩大255倍也仅仅有1...很黑

为了清楚显示，作了reverse操作，f[i]=255-f[i]，after reverse就比較明显，当中，白色亮区域是低概率位置，比較暗的是高概率位置。

最后，进行了一次二值化

二值化后黑色的区域就可能是我们的模版匹配的地方，可是，这种点太多了，改进的方法就是不用灰度投影，而採用彩色投影。

初识API

C++: void calcBackProject(const
Mat* images, int nimages, const int* channels, InputArray hist, OutputArray backProject, const float** ranges, double scale=1, bool uniform=true )

	images – Source arrays. They all should have the same depth, `CV_8U` or `CV_32F` , and the same size. Each of them can have an arbitrary number of channels. nimages – Number of source images. channels – The list of channels used to compute the back projection. The number of channels must match the histogram dimensionality. The first array channels are numerated from 0 to `images[0].channels()-1` , the second array channels are counted from `images[0].channels()` to `images[0].channels() + images[1].channels()-1`, and so on. hist – Input histogram that can be dense or sparse. backProject – Destination back projection array that is a single-channel array of the same size and depth as `images[0]` . ranges – Array of arrays of the histogram bin boundaries in each dimension. See `calcHist()` . scale – Optional scale factor for the output back projection. uniform – Flag indicating whether the histogram is uniform or not (see above).

images – Source arrays. They all should have the same depth, CV_8U or CV_32F ,
and the same size. Each of them can have an arbitrary number of channels.
nimages – Number of source images.
channels – The list of channels used to compute the back projection. The number of channels must match the histogram dimensionality. The first array channels are numerated from 0
to images[0].channels()-1 , the second array channels are counted from images[0].channels() to images[0].channels() + images[1].channels()-1,
and so on.
hist – Input histogram that can be dense or sparse.
backProject – Destination back projection array that is a single-channel array of the same size and depth as images[0] .
ranges – Array of arrays of the histogram bin boundaries in each dimension. See calcHist() .
scale – Optional scale factor for the output back projection.
uniform – Flag indicating whether the histogram is uniform or not (see above).

对于參数就不多作解释了，由于和计算直方图的參数差点儿一摸一样，scale是个扩大系数，常常取255.我们通过以下的样例来进一步巩固。

荷枪实弹

刚才的那3个效果图大家都看到了，我就先拿出灰度反向投影的代码

这段代码比較简单，由于Histogram1D这个类见了非常多次了

#include "opencv2/highgui/highgui.hpp"
#include "opencv2/imgproc/imgproc.hpp"
#include <iostream>
using namespace cv;
using namespace std;class Histogram1D {
private:
int histSize[1]; // number of bins
float hranges[2]; // min and max pixel value
const float* ranges[1];
int channels[1]; // only 1 channel used here
public:
Histogram1D() {
histSize[0]= 256;
hranges[0]= 0.0;
hranges[1]= 255.0;
ranges[0]= hranges;
channels[0]= 0;
}Mat getHistogram(const cv::Mat &image) {
Mat hist;
calcHist(&image,1,channels,Mat(),hist,1,histSize,ranges);
return hist;
}
};Mat applyLookUp(const cv::Mat& image,const cv::Mat& lookup) {
Mat result;
cv::LUT(image,lookup,result);
return result;
}int main( int, char** argv )
{
Mat image,gray,backproj;
image = imread( argv[1], 1 );
if( !image.data )
return -1;
cvtColor(image, gray, CV_BGR2GRAY);Mat imageROI;
imageROI= gray(cv::Rect(360,55,40,50)); // 模版图像Histogram1D h;
Mat hist= h.getHistogram(imageROI);
normalize(hist,hist,1.0); //归一化，得到概率直方图
float range[] = { 0, 255 };
const float* ranges = { range };
calcBackProject( &gray, 1, 0, hist, backproj, &ranges, 255.0); //反向投影，最重要的一步namedWindow("after project");
imshow("after project",backproj);Mat lut(1,256,CV_8U);
for (int i=0; i<256; i++) {
lut.at<uchar>(i)= 255-i; //reverse操作
}
Mat out = applyLookUp(backproj,lut);
namedWindow("after reverse");
imshow("after reverse",out);threshold(out, out, 240 ,255, THRESH_BINARY); // 以240为分界线,<240为0，否则为255
namedWindow("after threshold");
imshow("after threshold",out); waitKey(0);
return 0;
}

之前有提到过，灰度反向投影太不准确，非常多错误的点都被弄进来了，以下，我们实现彩色反向投影。

先介绍一个类，跟Histogram1D非常像，仅仅只是是3D了 0 0

class ColorHistogram {
private:
int histSize[3];
float hranges[2];
const float* ranges[3];
int channels[3]; public:ColorHistogram() {// Prepare arguments for a color histogram
histSize[0]= histSize[1]= histSize[2]= 256;
hranges[0]= 0.0; // BRG range
hranges[1]= 255.0;
ranges[0]= hranges; // all channels have the same range
ranges[1]= hranges;
ranges[2]= hranges;
channels[0]= 0;// the three channels
channels[1]= 1;
channels[2]= 2;
}// Computes the histogram.
Mat getHistogram(const Mat &image) {Mat hist;// BGR color histogram
hranges[0]= 0.0; // BRG range
hranges[1]= 255.0;
channels[0]= 0;// the three channels
channels[1]= 1;
channels[2]= 2; calcHist(&image, 1,channels,Mat(),hist,3,histSize,ranges);
return hist;
}Mat colorReduce(const Mat &image, int div=64) {
int n= (int)(log((double)(div))/log(2.0));
// mask used to round the pixel value
uchar mask= 0xFF<<n; // e.g. for div=16, mask= 0xF0 Mat_<Vec3b>::const_iterator it= image.begin<Vec3b>();
Mat_<Vec3b>::const_iterator itend= image.end<Vec3b>(); // Set output image (always 1-channel)
Mat result(image.rows,image.cols,image.type());
Mat_<Vec3b>::iterator itr= result.begin<Vec3b>(); for ( ; it!= itend; ++it, ++itr) {
(*itr)[0]= ((*it)[0]&mask) + div/2;
(*itr)[1]= ((*it)[1]&mask) + div/2;
(*itr)[2]= ((*it)[2]&mask) + div/2;
} return result;
}};

最须要注意的是colorReduce这种方法

假如div = 16,那么最后的单通道颜色仅仅有16种。由256变成了16，也就是本来的256^3,缩小成了16^3

我们处理彩色图像前，最好先对颜色作color reduce.

我的想法是，由于原来的色彩太多了，256^3,范围太大，直方图概率函数会非常小，可能就会发生难以预料的错误了，比方会造成基本同样的颜色却推断不一致。

我们再来看一个类

class ContentFinder {
private:
float hranges[2];
const float* ranges[3];
int channels[3];
float threshold;
Mat histogram;public:
ContentFinder() : threshold(-1.0f) {
hranges[0]= 0.0;
hranges[1]= 255.0;
channels[0]= 0;
channels[1]= 1;
channels[2]= 2;
ranges[0]= hranges;
ranges[1]= hranges;
ranges[2]= hranges;
}// Sets the threshold on histogram values [0,1]
void setThreshold(float t) {
threshold= t;
}// Gets the threshold
float getThreshold() {
return threshold;
}//设置了直方图，并归一化，生成概率直方图
void setHistogram(const Mat& h) {
histogram= h;
normalize(histogram,histogram,1.0);
}Mat find(const Mat& image) {
Mat result;
//最重要的一步，产生反向投影图
calcBackProject(&image, 1,channels,histogram,result,ranges,255.0);
//二值化
if (threshold>0.0)
cv::threshold(result, result,255*threshold, 255, cv::THRESH_BINARY);
return result;
}};

然后，看下我们的main函数

int main( int, char** argv )
{
Mat color;
color = imread( argv[1], 1 );
if( !color.data )
return -1; ColorHistogram hc;// reduce colors
color = hc.colorReduce(color,32);
// blue sky area
Mat imageROI = color(Rect(0,0,165,75));
Mat hist= hc.getHistogram(imageROI);
ContentFinder finder;
finder.setHistogram(hist);
finder.setThreshold(0.05f);Mat result= finder.find(color);namedWindow("sample");
imshow("sample",result); waitKey(0);
return 0;
}

我们来看一下效果，先看原图，我们选取了左上角的蓝天作模版图