Stereo Local Block Matching

Stereo block matching is a method to estimate the motion of the blocks between the consecutive frames, called stereo pair. The postulate behind this idea is that, considering a stereo pair, the foreground objects will have disparities higher than the background. Local block matching uses the information in the neighboring patch based on the window size, for identifying the conjugate point in its stereo pair. While, the techniques under global method, used the information from the whole image for computing the matching pixel, providing much better accuracy than local methods. But, the efficiency in the global methods are obtained with the cost of resources, which is where local methods stands out.

Local block matching algorithm consists of preprocessing and disparity estimation stages. The preprocessing consists of Sobel gradient computation followed by image clipping. And the disparity estimation consists of SAD (Sum of Absolute Difference) computation and obtaining the disparity using winner takes all method (least SAD will be the disparity). Invalidity of the pixel relies upon its uniqueness from the other possible disparities. And the invalid pixels are indicated with the disparity value of zero.

API Syntax

template void xFFindStereoCorrespondenceBM (xF::Mat & left_image, xF::Mat & right_image, xF::Mat & disparity_image, xF::SBMState< WSIZE,NDISP,NDISP_UNITS > &sbmstate)

Parameter Descriptions

The following table describes the template and the function parameters.

Table 1.xFFindStereoCorrespondenceBM Function Parameter Descriptions
Parameter Description
WSIZE Size of the window used for disparity computation
NDISP Number of disparities
NDISP_UNITS Number of disparities to be computed in parallel.
SRC_T Input pixel type. Only 8-bit, unsigned, 1 channel is supported (XF_8UC1)
DST_T Output type. This is XF_16UC1, where the disparities are arranged in Q12.4 format.
ROWS Maximum height of input and output image (must be a multiple of 8)
COLS Maximum width of input and output image (must be a multiple of 8)
NPC Number of pixels to be processed per cycle; possible options are XF_NPPC1 only.
left_image Image from the left camera
right_image Image from the right camera
disparity_image Disparities output in the form of an image.
sbmstate

Class object consisting of various parameters regarding the stereo block matching algorithm.

1. preFilterCap: default value is 31, can be altered by the user, value ranges from 1 to 63

2. minDisparity: default value is 0, can be altered by the user, value ranges from 0 to (imgWidth-NDISP)

3. uniquenessRatio: default set to 15, but can be altered to any non-negative integer.

4. textureThreshold: default set to 10, but can be modified to any non-negative integer.

Resource Utilization

The following table summarizes the resource utilization of the kernel in different configurations, generated usingVivado HLS 2017.1version tool for the Xilinx Xczu9eg-ffvb1156-1-i-es1 FPGA, to progress a grayscale HD (1080x1920) image.

The configurations are in the format: imageSize_SADwinSize_NDisp_NDispUnits.

Table 2.xFFindStereoCorrespondenceBM Function Resource Utilization Summary
Configurations

Frequency

( MHz)

 Resource Utilization
BRAM_18k DSP48E FF LUT
HD_5_16_2 300 37 20 6856 7181
HD_9_32_4 300 45 20 9700 10396
HD_11_32_32 300 49 20 34519 31978
HD_15_128_32 300 57 20 41017 35176
HD_21_64_16 300 69 20 29853 30706

Performance Estimate

The following table summarizes a performance estimate of the Stereo local block matching in different configurations, as generated usingVivado HLS 2017.1tool for Xilinx Xczu9eg-ffvb1156-1-i-es1 FPGA, to process a grayscale HD (1080x1920) image.

The configurations are in the format: imageSize_SADwinSize_NDisp_NDispUnits.

Table 3.xFFindStereoCorrespondenceBM Function Performance Estimate Summary
Configurations

Frequency

( MHz)

 Latency ( ms)
Min Max
HD_5_16_2 300 55.296 55.296
HD_9_32_4 300 55.296 55.296
HD_11_32_32 300 6.912 6.912
HD_15_48_16 300 20.736 20.736
HD_15_128_32 300 27.648 27.648
HD_21_64_16 300 27.648 27.648