Getting Started with SDSoC

This chapter provides the information you need to bring up your design using thexfOpenCVlibrary functions.

Prerequisites

This section lists the prerequisites for using the xfOpenCVlibrary functions on ZCU104 based platforms. The methodology holds true for ZC702 and ZC706 reVISION platforms as well.
  • Download and install the SDx development environment according to the directions provided inSDSoC Development Environment Release Notes, Installation, and Licensing Guide. Before launching the SDx development environment on Linux, set the$SYSROOTenvironment variable to point to the Linux root file system if using terminal to build project, delivered with the reVISION platform. For example:
    export SYSROOT = /zcu104_rv_ss/sw/a53_linux/a53_linux/sysroot/aarch64-xilinx-xilinx
  • Download theZynq® UltraScale+™ MPSoCEmbedded Vision Platform zip file and extract its contents. Create the SDx development environment workspace in thezcu104_rv_ssfolder of the extracted design file hierarchy. For more details, see thereVISION Getting Started Guide.
  • Set up the ZCU104 evaluation board. For more details, see thereVISION Getting Started Guide.
  • Download thexfOpenCVlibrary. This library is made available through github. Run the followinggit clonecommand to clone thexfOpenCVrepository to your local disk:
    git clone https://github.com/Xilinx/xfopencv.git

Migrating HLS Video Library to xfOpenCV

The HLS video library will soon be deprecated .All the functions and most of the infrastructure available in HLS video library are now available inxfOpenCVwith their names changed and some modifications. These HLS video library functions ported toxfOpenCVsupport SDSoc build flow also.

This section provides the details on using the C++ video processing functions and the infrastructure present in HLS video library.

Infrastructure Functions and Classes

All the functions imported from HLS video library now take xf::Mat (in sync withxfOpenCVlibrary) to represent image data instead of hls::Mat. The main difference between these two is that the hls::Mat uses hls::stream to store the data whereas xf::Mat uses a pointer. Therefore, hls:: Mat cannot be exactly replaced with xf::Mat for migrating.

Below table summarizes the differences between member functions of hls::Mat to xf::Mat.

Table 1.Infrastructure Functions and Classes
Member Function hls::Mat (HLS Video lib) Xf::Mat (xfOpenCV lib)
channels() Returns the number of channels Returns the number of channels
type() Returns the enum value of pixel type Returns the enum value of pixel type
depth() Returns the enum value of pixel type Returns the depth of pixel including channels
read() Readout a value and return it as a scalar from stream Readout a value from a given location and return it as a packed (for multi-pixel/clock) value.
operator >> Similar to read() Not available inxfOpenCV
operator << Similar to write() Not available inxfOpenCV
Write() Write a scalar value into the stream Writes a packed (for multi-pixel/clock) value into the given location..

Infrastructure files available in HLS Video Library hls_video_core.h, hls_video_mem.h, hls_video_types.h are moved to xf_video_core.h, xf_video_mem.h, xf_video_types.h inxfOpenCVLibrary and hls_video_imgbase.h is deprecated. Code inside these files unchanged except that these are now under xf::namespace.

Classes

Memory Window Buffer
hls::window is now xf::window. No change in the implementation, except the namespace change. This is located in “xf_video_mem.h” file.
Memory Line Buffer
hls::LineBuffer is now xf::LineBuffer. No difference between the two, except xf::LineBuffer has extra template arguments for inferring different types of RAM structures, for the storage structure used. Default storage type is “RAM_S2P_BRAM” with RESHAPE_FACTOR=1. Complete description can be found here xf::LineBuffer. This is located in xf_video_mem.hfile.

Funtions

OpenCV interface functions
These functions covert image data of OpenCV Mat format to/from HLS AXI types. HLS Video Library had 14 interface functions, out of which, two functions are available in xfOpenCV Library: cvMat2AXIvideo and AXIvideo2cvMat located in “xf_axi.h” file. The rest are all deprecated.
AXI4-Stream I/O Functions
The I/O functions which convert hls::Mat to/from AXI4-Stream compatible data type (hls::stream) are hls::AXIvideo2Mat, hls::Mat2AXIvideo. These functions are now deprecated and added 2 new functions xf::AXIvideo2xfMat and xf:: xfMat2AXIvideo to facilitate the xf::Mat to/from conversion. To use these functions, the header file "xf_infra.h" must be included.

xf::window

A template class to represent the 2D window buffer. It has three parameters to specify the number of rows, columns in window buffer and the pixel data type.

Class definition

template class Window { public: Window() /* Window main APIs */ void shift_pixels_left(); void shift_pixels_right(); void shift_pixels_up(); void shift_pixels_down(); void insert_pixel(T value, int row, int col); void insert_row(T value[COLS], int row); void insert_top_row(T value[COLS]); void insert_bottom_row(T value[COLS]); void insert_col(T value[ROWS], int col); void insert_left_col(T value[ROWS]); void insert_right_col(T value[ROWS]); T& getval(int row, int col); T& operator ()(int row, int col); T val[ROWS][COLS]; #ifdef __DEBUG__ void restore_val(); void window_print(); T val_t[ROWS][COLS]; #endif };

Parameter Descriptions

The following table lists the xf::Window class members and their descriptions.

Table 2.Window Function Parameter Descriptions
Parameter Description
Val 2-D array to hold the contents of buffer.

Member Function Description

Table 3.Member Function Description
Function Description
shift_pixels_left() Shift the window left, that moves all stored data within the window right, leave the leftmost column (col = COLS-1) for inserting new data.
shift_pixels_right() Shift the window right, that moves all stored data within the window left, leave the rightmost column (col = 0) for inserting new data.
shift_pixels_up() Shift the window up, that moves all stored data within the window down, leave the top row (row = ROWS-1) for inserting new data.
shift_pixels_down() Shift the window down, that moves all stored data within the window up, leave the bottom row (row = 0) for inserting new data.
insert_pixel(T value, int row, int col) Insert a new element value at location (row, column) of the window.
insert_row(T value[COLS], int row) Inserts a set of values in any row of the window.
insert_top_row(T value[COLS]) Inserts a set of values in the top row = 0 of the window.
insert_bottom_row(T value[COLS]) Inserts a set of values in the bottom row = ROWS-1 of the window.
insert_col(T value[ROWS], int col) Inserts a set of values in any column of the window.
insert_left_col(T value[ROWS]) Inserts a set of values in left column = 0 of the window.
insert_right_col(T value[ROWS]) Inserts a set of values in right column = COLS-1 of the window.
T& getval(int row, int col) Returns the data value in the window at position (row,column).
T& operator ()(int row, int col) Returns the data value in the window at position (row,column).
restore_val() Restore the contents of window buffer to another array.
window_print() Print all the data present in window buffer onto console.

Template Parameter Description

Table 4.Template Parameter Description
Parameter Description
ROWS Number of rows in the window buffer.
COLS Number of columns in the window buffer.
T Data type of pixel in the window buffer.
Sample code for window buffer declaration 
Window kernel;

xf::LineBuffer

A template class to represent 2D line buffer. It has three parameters to specify the number of rows, columns in window buffer and the pixel data type.

Class definition

template class LineBuffer { public: LineBuffer() /* LineBuffer main APIs */ /* LineBuffer main APIs */ void shift_pixels_up(int col); void shift_pixels_down(int col); void insert_bottom_row(T value, int col); void insert_top_row(T value, int col); void get_col(T value[ROWS], int col); T& getval(int row, int col); T& operator ()(int row, int col); /* Back compatible APIs */ void shift_up(int col); void shift_down(int col); void insert_bottom(T value, int col); void insert_top(T value, int col); T val[ROWS][COLS]; #ifdef __DEBUG__ void restore_val(); void linebuffer_print(int col); T val_t[ROWS][COLS]; #endif };

Parameter Descriptions

The following table lists the xf::LineBuffer class members and their descriptions.

Table 5.Line Buffer Function Parameter Descriptions
Parameter Description
Val 2-D array to hold the contents of line buffer.

Member Functions Description

Table 6.Member Functions Description
Function Description
shift_pixels_up(int col) Line buffer contents Shift up, new values will be placed in the bottom row=ROWS-1.
shift_pixels_down(int col) Line buffer contents Shift down, new values will be placed in the top row=0.
insert_bottom_row(T value, int col) Inserts a new value in bottom row= ROWS-1 of the line buffer.
insert_top_row(T value, int col) Inserts a new value in top row=0 of the line buffer.
get_col(T value[ROWS], int col) Get a column value of the line buffer.
T& getval(int row, int col) Returns the data value in the line buffer at position (row, column).
T& operator ()(int row, int col); Returns the data value in the line buffer at position (row, column).

Template Parameter Description

Table 7.Template Parameter Description
Parameter Description
ROWS Number of rows in line buffer.
COLS Number of columns in line buffer.
T Data type of pixel in line buffer.
MEM_TYPE Type of storage element. It takes one of the following enumerated values: RAM_1P_BRAM, RAM_1P_URAM, RAM_2P_BRAM, RAM_2P_URAM, RAM_S2P_BRAM, RAM_S2P_URAM, RAM_T2P_BRAM, RAM_T2P_URAM.
RESHAPE_FACTOR Specifies the amount to divide an array.
Sample code for line buffer declaration: 
LineBuffer<3, 1920, XF_8UC3, RAM_S2P_URAM,1> buff;

Video Processing Functions

The following table summarizes the video processing functions ported from HLS Video Library intoxfOpenCVLibrary along with the API modifications.

Table 8.Video Processing Functions
Functions HLS Video Library -API xfOpenCV Library-API
addS 
template void AddS(Mat&src,Scalar scl, Mat& dst)
 
template void addS(xf::Mat & _src1, unsigned char _scl[XF_CHANNELS(SRC_T,NPC)],xf::Mat & _dst)
AddWeighted 
template void AddWeighted(Mat& src1,P_T alpha,Mat& src2,P_T beta, P_T gamma,Mat& dst)
 
template< int SRC_T,int DST_T, int ROWS, int COLS, int NPC = 1> void addWeighted(xf::Mat & src1,float alpha, xf::Mat & src2,float beta, float gama, xf::Mat & dst)
Cmp 
template void Cmp(Mat& src1,Mat& src2, Mat& dst,int cmp_op)
 
template void compare(xf::Mat & _src1, xf::Mat & _src2,xf::Mat & _dst)
CmpS 
template void CmpS(Mat& src, P_T value, Mat& dst, int cmp_op)
 
template void compare(xf::Mat & _src1, unsigned char _scl[XF_CHANNELS(SRC_T,NPC)],xf::Mat & _dst)
Max 
template void Max(Mat& src1, Mat& src2, Mat& dst)
 
template void Max(xf::Mat & _src1, xf::Mat & _src2,xf::Mat & _dst)
MaxS 
template void MaxS(Mat& src, _T value, Mat& dst)
 
template< int SRC_T, int ROWS, int COLS, int NPC =1> void max(xf::Mat & _src1, unsigned char _scl[XF_CHANNELS(SRC_T,NPC)],xf::Mat & _dst)
Min 
template void Min(Mat& src1, Mat& src2, Mat& dst)
 
template< int SRC_T, int ROWS, int COLS, int NPC =1> void Min(xf::Mat & _src1, xf::Mat & _src2,xf::Mat & _dst)
MinS 
template void MinS(Mat& src, _T value,Mat& dst)
 
template< int SRC_T, int ROWS, int COLS, int NPC =1> void min(xf::Mat & _src1, unsigned char _scl[XF_CHANNELS(SRC_T,NPC)],xf::Mat & _dst)
PaintMask 
template void PaintMask( Mat &_src, Mat&_mask, Mat&_dst,Scalar _color)
 
template< int SRC_T,int MASK_T, int ROWS, int COLS,int NPC=1> void paintmask(xf::Mat & _src_mat, xf::Mat & in_mask, xf::Mat & _dst_mat, unsigned char _color[XF_CHANNELS(SRC_T,NPC)])
Reduce 
template void Reduce( Mat &src, Mat &dst, int dim, int op=HLS_REDUCE_SUM)
 
template< int REDUCE_OP, int SRC_T,int DST_T, int ROWS, int COLS,int ONE_D_HEIGHT, int ONE_D_WIDTH, int NPC=1> void reduce(xf::Mat & _src_mat, xf::Mat & _dst_mat, unsigned char dim)
Zero 
template void Zero(Mat& src, Mat& dst)
 
template< int SRC_T, int ROWS, int COLS, int NPC =1> void zero(xf::Mat & _src1,xf::Mat & _dst)
Sum 
template Scalar Sum( Mat& src)
 
template< int SRC_T, int ROWS, int COLS, int NPC = 1> void sum(xf::Mat & src1, double sum[XF_CHANNELS(SRC_T,NPC)] )
SubS 
template void SubS(Mat& src, Scalar scl, Mat& dst)
 
template void SubS(xf::Mat & _src1, unsigned char _scl[XF_CHANNELS(SRC_T,NPC)],xf::Mat & _dst)
SubRS 
template void SubRS(Mat& src, Scalar scl, Mat& dst)
 
template void SubRS(xf::Mat & _src1, unsigned char _scl[XF_CHANNELS(SRC_T,NPC)],xf::Mat & _dst)
Set 
template void Set(Mat& src, Scalar scl, Mat& dst)
 
template< int SRC_T, int ROWS, int COLS, int NPC =1> void set(xf::Mat & _src1, unsigned char _scl[XF_CHANNELS(SRC_T,NPC)],xf::Mat & _dst)
Absdiff 
template void AbsDiff( Mat& src1, Mat& src2, Mat& dst)
 
template void absdiff(xf::Mat & _src1,xf::Mat & _src2,xf::Mat & _dst)
And 
template void And( Mat& src1, Mat& src2, Mat& dst)
 
template void bitwise_and(xf::Mat & _src1, xf::Mat & _src2, xf::Mat &_dst)
Dilate 
template void Dilate(Mat&_src,Mat&_kernel)
 
template void dilate (xf::Mat & _src, xf::Mat & _dst,unsigned char _kernel[K_ROWS*K_COLS])
Duplicate 
template void Duplicate(Mat& src,Mat& dst1,Mat& dst2)
 
template void duplicateMat(xf::Mat & _src, xf::Mat & _dst1,xf::Mat & _dst2)
EqualizeHist 
template void EqualizeHist(Mat&_src,Mat&_dst)
 
template void equalizeHist(xf::Mat & _src,xf::Mat & _src1,xf::Mat & _dst)
erode 
template void Erode(Mat&_src,Mat&_dst,Window&_kernel)
 
template void erode (xf::Mat & _src, xf::Mat & _dst,unsigned char _kernel[K_ROWS*K_COLS])
FASTX 
template void FASTX(Mat &_src, Mat&_mask,HLS_TNAME(SRC_T)_threshold,bool _nomax_supression)
 
template void fast(xf::Mat & _src_mat,xf::Mat & _dst_mat,unsigned char _threshold)
Filter2D 
template void Filter2D(Mat &_src,Mat &_dst,Window&_kernel,Point_anchor)
 
template void filter2D(xf::Mat & _src_mat,xf::Mat & _dst_mat,short int filter[FILTER_HEIGHT*FILTER_WIDTH],unsigned char _shift)
GaussianBlur 
template void GaussianBlur(Mat &_src, Mat &_dst,double sigmaX=0,double sigmaY=0)
 
template void GaussianBlur(xf::Mat & _src, xf::Mat & _dst, float sigma)
Harris 
template void Harris(Mat &_src,Mat&_dst,KT k,int threshold
 
template void cornerHarris(xf::Mat & src,xf::Mat & dst,uint16_t threshold, uint16_t k)
CornerHarris 
template void CornerHarris( Mat&_src,Mat&_dst,KT k)
 
template void cornerHarris(xf::Mat & src,xf::Mat & dst,uint16_t threshold, uint16_t k
HoughLines2 
template void HoughLines2(Mat &_src, Polar_ (&_lines)[linesMax],unsigned int threshold)
 
template void HoughLines(xf::Mat & _src_mat,float outputrho[MAXLINES],float outputtheta[MAXLINES],short threshold,short linesmax)
Integral 
template void Integral(Mat&_src, Mat&_sum )
 
template void integral(xf::Mat & _src_mat, xf::Mat & _dst_mat)
Merge 
template void Merge( Mat& src0, Mat& src1, Mat& src2, Mat& src3, Mat& dst)
 
template void merge(xf::Mat &_src1, xf::Mat &_src2, xf::Mat &_src3, xf::Mat &_src4, xf::Mat &_dst)
MinMaxLoc 
template void MinMaxLoc(Mat& src, P_T* min_val,P_T* max_val,Point& min_loc, Point& max_loc)
 
template void minMaxLoc(xf::Mat & _src,int32_t *min_value, int32_t *max_value,uint16_t *_minlocx, uint16_t *_minlocy, uint16_t *_maxlocx, uint16_t *_maxlocy )
Mul 
template void Mul(Mat& src1, Mat& src2, Mat& dst)
 
template void multiply(xf::Mat & src1, xf::Mat & src2, xf::Mat & dst,float scale)
Not 
template void Not(Mat& src, Mat& dst)
 
template void bitwise_not(xf::Mat & src, xf::Mat & dst)
Range 
template void Range(Mat& src, Mat& dst, P_T start,P_T end)
 
template void inRange(xf::Mat & src,unsigned char lower_thresh,unsigned char upper_thresh,xf::Mat & dst)
Resize 
template void Resize ( Mat &_src, Mat &_dst, int interpolation=HLS_INTER_LINEAR )
 
template void resize (xf::Mat & _src, xf::Mat & _dst)
sobel 
template void Sobel (Mat &_src,Mat &_dst)
 
template void Sobel(xf::Mat & _src_mat,xf::Mat & _dst_matx,xf::Mat & _dst_maty)
split 
template void Split( Mat& src, Mat& dst0, Mat& dst1, Mat& dst2, Mat& dst3)
 
template void extractChannel(xf::Mat & _src_mat, xf::Mat & _dst_mat, uint16_t _channel)
Threshold 
template void Threshold( Mat& src, Mat& dst, HLS_TNAME(SRC_T) thresh, HLS_TNAME(DST_T) maxval, int thresh_type)
 
template void Threshold(xf::Mat & _src_mat,xf::Mat & _dst_mat,short int thresh,short int maxval )
Scale 
template void Scale(Mat& src,Mat& dst, P_T scale=1.0,P_T shift=0.0)
 
template< int SRC_T,int DST_T, int ROWS, int COLS, int NPC = 1> void scale(xf::Mat & src1, xf::Mat & dst,float scale, float shift)
InitUndistortRectifyMapInverse 
template void InitUndistortRectifyMapInverse ( Window<3,3, CMT> cameraMatrix,DT(&distCoeffs)[N],Window<3,3, ICMT> ir, Mat &map1,Mat &map2,int noRotation=false)
 
template< int CM_SIZE, int DC_SIZE, int MAP_T, int ROWS, int COLS, int NPC > void InitUndistortRectifyMapInverse ( ap_fixed<32,12> *cameraMatrix, ap_fixed<32,12> *distCoeffs, ap_fixed<32,12> *ir, xf::Mat &_mapx_mat,xf::Mat &_mapy_mat,int _cm_size, int _dc_size)
Avg, mean, AvgStddev 
template DST_T Mean(Mat& src)
 
templatevoid meanStdDev(xf::Mat & _src,unsigned short* _mean,unsigned short* _stddev)
CvtColor 
template void CvtColor(Mat &_src, Mat &_dst)
 Color Conversion
Note:All the functions except Reduce can process N-pixels per clock where N is power of 2.

Using the xfOpenCV Library

This section describes using thexfOpenCVlibrary in the SDx development environment.

Note:The instructions in this section assume that you have downloaded and installed all the required packages. For more information, see the Prerequisites.

includefolder constitutes all the necessary components to build a Computer Vision or Image Processing pipeline using the library. The folderscommonandcorecontain the infrastructure that the library functions need for basic functions, Mat class, and macros. The library functions are categorized into three folders,features,videoandimgprocbased on the operation they perform. The names of the folders are self-explanatory.

To work with the library functions, you need to include the path to the The xfOpenCV library is structured as shown in the following table. Theincludefolder in the SDx project. You can include relevant header files for the library functions you will be working with after you source theincludefolder’s path to the compiler. For example, if you would like to work with Harris Corner Detector and Bilateral Filter, you must use the following lines in the host code:

#include “features/xf_harris.hpp” //for Harris Corner Detector #include “imgproc/xf_bilateral_filter.hpp” //for Bilateral Filter #include “video/xf_kalmanfilter.hpp”

After the headers are included, you can work with the library functions as described in thexfOpenCV Library API Referenceusing the examples in theexamplesfolder as reference.

The following table gives the name of the header file, including the folder name, which contains the library function.

Table 9.xfOpenCVLibrary Contents
Function Name File Path in theincludefolder
xf::accumulate imgproc/xf_accumulate_image.hpp
xf::accumulateSquare imgproc/xf_accumulate_squared.hpp
xf::accumulateWeighted imgproc/xf_accumulate_weighted.hpp
The xfOpenCV library is structured as shown in the following table.xf::absdiff, xf::add, xf::subtract, xf::bitwise_and, xf::bitwise_or, xf::bitwise_not, xf::bitwise_xor,xf::multiply ,xf::Max, xf::Min, xf::compare, xf::zero, xf::addS, xf::SubS, xf::SubRS ,xf::compareS, xf::MaxS, xf::MinS, xf::set core/xf_arithm.hpp
xf::addWeighted imgproc/xf_add_weighted.hpp
xf::bilateralFilter imgproc/xf_histogram.hpp
xf::boxFilter imgproc/xf_box_filter.hpp
xf::boundingbox imgproc/xf_boundingbox.hpp
xf::Canny imgproc/xf_canny.hpp
xf::Colordetect imgproc/xf_colorthresholding.hpp, imgproc/xf_bgr2hsv.hpp, imgproc/xf_erosion.hpp, imgproc/xf_dilation.hpp
xf::merge imgproc/xf_channel_combine.hpp
xf::extractChannel imgproc/xf_channel_extract.hpp
xf::convertTo imgproc/xf_convert_bitdepth.hpp
xf::crop imgproc/xf_crop.hpp
xf::filter2D imgproc/xf_custom_convolution.hpp
xf::nv122iyuv, xf::nv122rgba, xf::nv122yuv4, xf::nv212iyuv, xf::nv212rgba, xf::nv212yuv4, xf::rgba2yuv4, xf::rgba2iyuv, xf::rgba2nv12, xf::rgba2nv21, xf::uyvy2iyuv, xf::uyvy2nv12, xf::uyvy2rgba, xf::yuyv2iyuv, xf::yuyv2nv12, xf::yuyv2rgba,xf::rgb2iyuv,xf::rgb2nv12,xf::rgb2nv21,xf::rgb2yuv4,xf::rgb2uyvy,xf::rgb2yuyv,xf::rgb2bgr,xf::bgr2uyvy,xf::bgr2yuyv,xf::bgr2rgb,xf::bgr2nv12,xf::bgr2nv21,xf::iyuv2nv12,xf::iyuv2rgba,xf::iyuv2rgb,xf::iyuv2yuv4,xf::nv122uyvy,xf::nv122yuyv,xf::nv122nv21,xf::nv212rgb,xf::nv212bgr,xf::nv212uyvy,xf::nv212yuyv,xf::nv212nv12,xf::uyvy2rgb,xf::uyvy2bgr,xf::uyvy2yuyv,xf::yuyv2rgb,xf::yuyv2bgr,xf::yuyv2uyvy,xf::rgb2gray,xf::bgr2gray,xf::gray2rgb,xf::gray2bgr,xf::rgb2xyz,xf::bgr2xyz... imgproc/xf_cvt_color.hpp
xf::dilate imgproc/xf_dilation.hpp
xf::demosaicing imgproc/xf_demosaicing.hpp
xf::erode imgproc/xf_erosion.hpp
xf::fast features/xf_fast.hpp
xf::GaussianBlur imgproc/xf_gaussian_filter.hpp
xf::cornerHarris features/xf_harris.hpp
xf::calcHist imgproc/xf_histogram.hpp
xf::equalizeHist imgproc/xf_hist_equalize.hpp
xf::HOGDescriptor imgproc/xf_hog_descriptor.hpp
xf::Houghlines imgproc/xf_houghlines.hpp
xf::inRange imgproc/xf_inrange.hpp
xf::integralImage imgproc/xf_integral_image.hpp
xf::densePyrOpticalFlow video/xf_pyr_dense_optical_flow.hpp
xf::DenseNonPyrLKOpticalFlow video/xf_dense_npyr_optical_flow.hpp
xf::LUT imgproc/xf_lut.hpp
xf::KalmanFilter video/xf_kalmanfilter.hpp
xf::magnitude core/xf_magnitude.hpp
xf::MeanShift imgproc/xf_mean_shift.hpp
xf::meanStdDev core/xf_mean_stddev.hpp
xf::medianBlur imgproc/xf_median_blur.hpp
xf::minMaxLoc core/xf_min_max_loc.hpp
xf::OtsuThreshold imgproc/xf_otsuthreshold.hpp
xf::phase core/xf_phase.hpp
xf::paintmask imgproc/xf_paintmask.hpp
xf::pyrDown imgproc/xf_pyr_down.hpp
xf::pyrUp imgproc/xf_pyr_up.hpp
xf::reduce imgrpoc/xf_reduce.hpp
xf::remap imgproc/xf_remap.hpp
xf::resize imgproc/xf_resize.hpp
xf::scale imgproc/xf_scale.hpp
xf::Scharr imgproc/xf_scharr.hpp
xf::SemiGlobalBM imgproc/xf_sgbm.hpp
xf::Sobel imgproc/xf_sobel.hpp
xf::StereoPipeline imgproc/xf_stereo_pipeline.hpp
xf::sum imgproc/xf_sum.hpp
xf::StereoBM imgproc/xf_stereoBM.hpp
xf::SVM imgproc/xf_svm.hpp
xf::Threshold imgproc/xf_threshold.hpp
xf::warpTransform imgproc/xf_warp_transform.hpp

The different ways to use the xfOpenCV library examples are listed below:

Downloading and Using xfOpenCV Libraries from SDx GUI

You can downloadxfOpenCVdirectly from SDx GUI. To build a project using the example makefiles on the Linux platform:

  1. From SDx IDE, clickXilinxand selectSDx Libraries.
  2. ClickDownloadnext to theXilinx xfOpenCV Library.

    Figure:SDx Libraries

    The library is downloaded into /Xilinx/SDx/2019.1/xfopencv. After the library is downloaded, the entire set of examples in the library are available in the list of templates while creating a new project.
    Note:The library can be added to any project from the IDE menu options.
  3. To add a library to a project, from SDx IDE, clickXilinxand selectSDx Libraries.
  4. SelectXilinx xfOpenCV Libraryand clickAdd to project. The dropdown menu consists of options of which project the libraries need to be included to.

All the headers as part of theinclude/folder inxfOpenCVlibrary would be copied into the local project directory as/libs/xfopencv/include. All the settings required for the libraries to be run are also set when this action is completed.

Building a Project Using the Example Makefiles on Linux

Use the following steps to build a project using the example makefiles on the Linux platform:

  1. Open a terminal.
  2. When building for revision platform, set the environment variable SYSROOT to <the path to platform folder>/sw/a53_linux/a53_linux/sysroot/aarch64-xilinx-linux.
  3. Change the platform variable to point to the downloaded platform folder in makefile. Ensure that the folder name of the downloaded platform is unchanged.
  4. When building for revision platform , change IDIRS and LDIRS variables in the Makefile as follows:
    IDIRS = -I. -I${SYSROOT}/usr/include -I ../../include LDIRS = --sysroot=${SYSROOT} -L=/lib -L=/usr/lib -Wl,-rpath-link=${SYSROOT}/lib,-rpath-link=${SYSROOT}/usr/lib
  5. Change the directory to the location where you want to build the example.
    cd
  6. When building for revision platform , add#include"opencv2/imgcodecs/imgcodecs.hpp"inxf_headers.hfile ,both in if and else part.
  7. Set the environment variables to run SDx development environment.
    • For c shell:
      source /settings.csh
    • For bash shell:
      source /settings.sh
  8. Type themakecommand in the terminal. Thesd_cardfolder is created and can be found in thefolder.
Note:Ignore 2,4 and 6 steps when building for Non revision platforms.

Using reVISION Samples on the reVISION Platform

Use the following steps to run a unit test for bilateral filter on zcu104_rv_ss:

  1. Launch the SDx development environment using the desktop icon or theStartmenu.

    TheWorkspace Launcherdialog appears.

  2. ClickBrowseto enter a workspace folder used to store your projects (you can use workspace folders to organize your work), then clickOKto dismiss theWorkspace Launcherdialog.
    Note:Before launching the SDx IDE on Linux, ensure that you use the same shell that you have used to set the $SYSROOTenvironment variable. This is usually the file path to the Linux root file system.

    The SDx development environment window opens with theWelcometab visible when you create a new workspace. TheWelcometab can be closed by clicking theXicon or minimized if you do not wish to use it.

  3. SelectFile>New>Xilinx SDx Projectfrom the SDx development environment menu bar.

    TheNew Projectdialog box opens.

  4. Specify the name of the project. For exampleBilateral.
  5. ClickNext.

    The theChoose Hardware Platformpage appears.

  6. From theChoose Hardware Platformpage, click theAdd Custom Platformbutton.
  7. Browse to the directory where you extracted the reVISION platform files. Ensure that you select thezcu104_rv_ssfolder.
  8. From theChoose Hardware Platformpage, selectzcu104_rv_ss (custom).
  9. ClickNext.

    TheTemplatespage appears, containing source code examples for the selected platform.

  10. From the list of application templates, selectbilateral - File I/Oand clickFinish.
  11. add#include"opencv2/imgcodecs/imgcodecs.hpp"inxf_headers.hfile present underproject/src/examples/bilateral/,both in if and else part.
  12. Click theActive build configurationsdrop-down from theSDx Project Settingswindow, to select the active configuration or create a build configuration.

    The standard build configurations are Debug and Release. To get the best runtime performance, switch to use theReleasebuild configuration as it uses a higher compiler optimization setting than the Debug build configuration.

    Figure:SDx Project Settings - Active Build Configuration



  13. Set theData motion network clock frequency (MHz)to the required frequency, on theSDx Project Settingspage.
  14. Right-click the project and selectBuild Projector press Ctrl+B keys to build the project, in theProject Explorerview.
  15. Copy the contents of the newly createdsd_cardfolder to the SD card. Thesd_cardfolder contains all the files required to run designs on theZCU104board.
  16. Insert the SD card in theZCU104board card slot and switch it ON.
    Note:A serial port emulator (Teraterm/ minicom) is required to interface the user commands to the board.
  17. Upon successful boot, run the following command in the Teraterm terminal (serial port emulator.)
    #cd /media/card #remount
  18. Run the.elffile for the respective functions.

    For more information, see theUsing the xfOpenCV Library Functions on Hardware.

Using the xfOpenCV Library on a non-reVISION Platform

This section describes using thexfOpenCVlibrary on a non-reVISION platform, in theSDx™development environment. The examples in xfOpenCV requireOpenCVlibraries for successful compilation. As non-reVISION platform may or may not contain opencv libs, as a perquisites it is required to install/compile opencv libraries(with compatible libjpeg.so).

Note:The instructions in this section assume that you have downloaded and installed all the required packages. For more information, see the Prerequisites.

Use the following steps to import thexfOpenCVlibrary into aSDxproject and execute it on a custom platform:

  1. Launch theSDxdevelopment environment using the desktop icon or theStartmenu.

    TheWorkspace Launcherdialog appears.

  2. ClickBrowseto enter a workspace folder used to store your projects (you can use workspace folders to organize your work), then clickOKto dismiss theWorkspace Launcherdialog.

    TheSDxdevelopment environment window opens with theWelcometab visible when you create a new workspace. TheWelcometab can be closed by clicking theXicon or minimized if you do not wish to use it.

  3. SelectFile>New>Xilinx SDx Projectfrom theSDxdevelopment environment menu bar.

    TheNew Projectdialog box opens.

  4. Specify the name of the project. For exampleTest.

  5. ClickNext.

    The theChoose Hardware Platformpage appears.

  6. From theChoose Hardware Platformpage, select a suitable platform. For example,zcu102.

  7. ClickNext.

    TheChoose Software Platform and Target CPUpage appears.

  8. From theChoose Software Platform and Target CPUpage, select an appropriate software platform and the target CPU. For example, selectA9from theCPUdropdown list for ZC702 and ZC706 reVISION platforms.
  9. ClickNext. TheTemplatespage appears, containing source code examples for the selected platform.

  10. From the list of application templates, selectEmpty Applicationand clickFinish.

    TheNew Projectdialog box closes. A new project with the specified configuration is created. TheSDx Project Settingsview appears. Notice the progress bar in the lower right border of the view, Wait for a few moments for theC/C++ Indexerto finish.

  11. The standard build configurations are Debug and Release. To get the best run-time performance, switch to use the Releasebuild configuration as it uses a higher compiler optimization setting than the Debug build configuration.

    Figure:SDx Project Settings - Active Build Configuration



  12. Set theData motion network clock frequency (MHz)to the required frequency, on theSDx Project Settingspage.
  13. Select theGenerate bitstreamandGenerate SD card imagecheck boxes.
  14. Right-click on the newly created project in theProject Explorerview.

  15. From the context menu that appears, selectC/C++ Build Settings.

    TheProperties for dialog box appears.

  16. Click theTool Settingstab.
  17. Expand theSDS++ Compiler>Directoriestree.

  18. Click theicon to add the"\include"and"\include"folder locations to theInclude Pathslist.

    Note:The OpenCVlibrary is not provided by Xilinx for custom platforms. You are required to provide the library. Use the reVISION platform in order to use the OpenCVlibrary provided by Xilinx.

    Figure:SDS++ Compiler Settings



  19. In the same page, underSDS++ Compiler>Inferred Options>Software Platform, specify "-hls-target 1" in the Software Platform Inferred Flags.
  20. ClickApply.
  21. Expand theSDS++ Linker>Librariestree.
  22. Click theicon and add the following libraries to theLibraries(-l)list. These libraries are required byOpenCV.
    • opencv_core
    • opencv_imgproc
    • opencv_imgcodecs
    • opencv_features2d
    • opencv_calib3d
    • opencv_flann
    • opencv_video
    • opencv_videoio

  23. Click theicon and add/libfolder location to theLibraries search path (-L)list.

    Note:The OpenCV library is not provided by Xilinx for custom platforms. You are required to provide the library. Use the reVISION platform in order to use the OpenCV library provided by Xilinx.

    Figure:SDS++ Linker Settings



  24. ClickApplyto save the configuration.
  25. ClickOKto close theProperties for dialog box.
  26. Expand the newly created project tree in theProject Explorerview.
  27. Right-click thesrcfolder and selectImport. TheImportdialog box appears.
  28. SelectFile Systemand clickNext.
  29. ClickBrowseto navigate to the/examplesfolder location.

  30. Select the folder that corresponds to the library that you desire to import. For example,accumulate.

    Figure:Import Library Example Source Files



  31. Right-click the library function in theProject Explorerview and selectToggle HW/SWto move the function to the hardware.

    Figure:Moving a Library Function to the Hardware



  32. Right-click the project and selectBuild Projector press Ctrl+B keys to build the project, in theProject Explorerview.

    The build process may take anytime between few minutes to several hours, depending on the power of the host machine and the complexity of the design. By far, the most time is spent processing the routines that have been tagged for realization in hardware.

  33. Copy the contents of the newly created.\\\Release\sd_cardfolder to the SD card. Thesd_cardfolder contains all the files required to run designs on a board.

  34. Insert the SD card in the board card slot and switch it ON.

    Note:A serial port emulator (Teraterm/ minicom) is required to interface the user commands to the board.
  35. Upon successful boot, navigate to the./mntfolder and run the following command at the prompt:
    #cd /mnt
    Note:It is assumed that the OpenCV libraries are a port of the root filesystem. If not, add the location of OpenCV libraries to LD_LIBRARY_PATHusing the $ export LD_LIBRARY_PATH=/libcommand.
  36. Run the.elfexecutable file. For more information, see theUsing the xfOpenCV Library Functions on Hardware.

Changing the Hardware Kernel Configuration

Use the following steps to change the hardware kernel configuration:
  1. Update thexfOpenCVgit folder>/xfOpenCV/examples//xf_config_params.hfile.
  2. Update the makefile along with thexf_config_params.hfile:
    1. Find the line with the function name in the makefile. For bilateral filter, the line in the makefile will bexf::BilateralFilter<3,1,0,1080,1920,1>.
    2. Update the template parameters in the makefile to reflect changes made in thexf_config_params.hfile. For more details, see thexfOpenCV Library API Reference.

Using the xfOpenCV Library Functions on Hardware

The following table lists thexfOpenCVlibrary functions and the command to run the respective examples on hardware. It is assumed that your design is completely built and the board has booted up correctly.

Table 10.Using the xfOpenCV Library Function on Hardware
Example Function Name Usage on Hardware
accumulate xf::accumulate ./.elf
accumulatesquared xf::accumulateSquare ./.elf
accumulateweighted xf::accumulateWeighted ./.elf
addS xf::addS ./.elf
arithm xf::absdiff, xf::add, xf::subtract, xf::bitwise_and, xf::bitwise_or, xf::bitwise_not, xf::bitwise_xor ./.elf
addweighted xf::addWeighted ./.elf
Bilateralfilter xf::bilateralFilter ./.elf
Boxfilter xf::boxFilter ./.elf
Boundingbox xf::boundingbox ./.elf
Canny xf::Canny ./.elf
channelcombine xf::merge ./.elf
Channelextract xf::extractChannel ./.elf
Colordetect xf::bgr2hsv, xf::colorthresholding, xf:: erode, and xf:: dilate ./.elf
compare xf::compare ./.elf
compareS xf::compareS ./.elf
Convertbitdepth xf::convertTo ./.elf
Cornertracker xf::cornerTracker ./exe
crop xf::crop ./.elf
Customconv xf::filter2D ./.elf
cvtcolor IYUV2NV12 xf::iyuv2nv12 ./.elf
cvtcolor IYUV2RGBA xf::iyuv2rgba ./.elf
cvtcolor IYUV2YUV4 xf::iyuv2yuv4 ./.elf
cvtcolor NV122IYUV xf::nv122iyuv ./.elf
cvtcolor NV122RGBA xf::nv122rgba ./.elf
cvtcolor NV122YUV4 xf::nv122yuv4 ./.elf
cvtcolor NV212IYUV xf::nv212iyuv ./.elf
cvtcolor NV212RGBA xf::nv212rgba ./.elf
cvtcolor NV212YUV4 xf::nv212yuv4 ./.elf
cvtcolor RGBA2YUV4 xf::rgba2yuv4 ./.elf
cvtcolor RGBA2IYUV xf::rgba2iyuv ./.elf
cvtcolor RGBA2NV12 xf::rgba2nv12 ./.elf
cvtcolor RGBA2NV21 xf::rgba2nv21 ./.elf
cvtcolor UYVY2IYUV xf::uyvy2iyuv ./.elf
cvtcolor UYVY2NV12 xf::uyvy2nv12 ./.elf
cvtcolor UYVY2RGBA xf::uyvy2rgba ./.elf
cvtcolor YUYV2IYUV xf::yuyv2iyuv ./.elf
cvtcolor YUYV2NV12 xf::yuyv2nv12 ./.elf
cvtcolor YUYV2RGBA xf::yuyv2rgba ./.elf
Demosaicing xf::demosaicing ./.elf
Difference of Gaussian xf:: GaussianBlur, xf:: duplicateMat, xf:: delayMat, and xf::subtract ./.elf
Dilation xf::dilate ./.elf
Erosion xf::erode ./.elf
Fast xf::fast ./.elf
Gaussianfilter xf::GaussianBlur ./.elf
Harris xf::cornerHarris ./.elf
Histogram xf::calcHist ./.elf
Histequialize xf::equalizeHist ./.elf
Hog xf::HOGDescriptor ./.elf
Houghlines xf::HoughLines ./.elf
inRange xf::inRange ./.elf
Integralimg xf::integralImage ./.elf
Lkdensepyrof xf::densePyrOpticalFlow ./.elf
Lknpyroflow xf::DenseNonPyrLKOpticalFlow ./.elf
Lut xf::LUT ./.elf
Kalman Filter xf::KalmanFilter ./.elf
Magnitude xf::magnitude ./.elf
Max xf::Max ./.elf
MaxS xf::MaxS ./.elf
meanshifttracking xf::MeanShift ./.elf
meanstddev xf::meanStdDev ./.elf
medianblur xf::medianBlur ./.elf
Min xf::Min ./.elf
MinS xf::MinS ./.elf
Minmaxloc xf::minMaxLoc ./.elf
otsuthreshold xf::OtsuThreshold ./.elf
paintmask xf::paintmask ./.elf
Phase xf::phase ./.elf
Pyrdown xf::pyrDown ./.elf
Pyrup xf::pyrUp ./.elf
reduce xf::reduce ./.elf
remap xf::remap ./.elf
Resize xf::resize ./.elf
scale xf::scale ./.elf
scharrfilter xf::Scharr ./.elf
set xf::set ./.elf
SemiGlobalBM xf::SemiGlobalBM ./.elf
sobelfilter xf::Sobel ./.elf
stereopipeline xf::StereoPipeline ./.elf
stereolbm xf::StereoBM ./.elf
subRS xf::SubRS ./.elf
subS xf::SubS ./.elf
sum xf::sum ./.elf
Svm xf::SVM ./.elf
threshold xf::Threshold ./.elf
warptransform xf::warpTransform ./.elf
zero xf::zero ./.elf