RZ/V2H
Software | License |
---|---|
OpenCV Accelerator | 3-clause BSD |
DRP Driver | GPL-2.0 |
Abbreviation | Full form |
---|---|
BSP | Board support package |
DRP | Dynamically Reconfigurable Processor |
OpenCV | Open-source Computer Vision library |
OpenCVA | OpenCV Accelerator |
1. Overview
2. Usage of OpenCV Accelerator
3. OpenCVA Function List
4. OpenCVA API specification and condition for using DRP
5. API functions to control OpenCVA
6. DRP conflict
This manual explains the OpenCV Accelerator in RZ/V2H Linux. OpenCV Accelerator performs OpenCV function in high performance without changing OpenCV API.
OpenCV Accelerator (referred to below as OpenCVA) can perform OpenCV functions using Dynamically Reconfigurable Processor (=DRP).
If DRP is enabled and the parameters of OpenCV meet the conditions, OpenCV is executed using DRP. If DRP is disabled or the parameters do not match, then OpenCV is executed using CPU. If the DRP is used, OpenCV will be executed at high performance. The performance depends on the parameters.
The output OpenCVA using DRP is almost identical to output OpenCV using CPU, but not exactly the same. Much of this difference is due to the accuracy of in the DRP's arithmetic unit, and some is due to differences in algorithms.
DRP is also used for MPEG4 encoding or decoding, however since DRP is a common HW resource, OpenCVA and MPEG4 cannot be used at the same time. For the same reason, OpenCVA must be used in single process and single thread. In case of conflicting use of DRP, it causes an exception errors or performance degradation.
For details, please see Chapter 5.2 and Chapter 6.
None.
OpenCVA is based on as follows version of OpenCV.
OpenCV 4.1.0-r0
For datails, please see OpenCV Document
You can use OpenCVA same as OpenCV as usual and you do not need to consider of OpenCVA architecture. OpenCVA is automatically executed by DRP as follows if it matches the conditions under which DRP can be used. For the DRP using conditions, see Chapter 4.
OpenCVA can disable DRP, for each function. See Chapter 5.1 for details.
The following table lists the OpenCV functions that can be executed using DRP in the OpenCVA.
OpenCV function name | Function |
---|---|
4.1. resize | Image Resize. |
4.2. cvtColor | Change color space. |
4.3. cvtColorTwoPlane | Change color space. |
4.4. GaussianBlur | Gaussian filter process. |
4.5. dilate | Areas of bright regions grow. |
4.6. erode | Areas of dark regions grow. |
4.7. morphologyEX | Combination of dilate and erode. |
4.8. filter2D | Image convolving. |
4.9. Sobel | Extracting image edges. |
4.10. adaptiveThreshold | Transforms a grayscale image to a binary image according to the formulae. |
4.11. matchTemplate | Compares a template against overlapped image regions. |
4.12. wrapAffine | Transforms the source image using the 2x3 matrix. |
4.13. wrapPerspective | Transforms the source image using the 3x3 matrix. |
4.14. pyrDown | Downsampling step of the Gaussian pyramid construction. |
4.15. pyrUp | Upsampling step of the Gaussian pyramid construction. |
4.16. FAST | Detects corners using the FAST algorithm. |
This chapter describes the OpenCV API that can be executed by DRP, and their conditions for using DRP.
Resize an image, set desired scale of each axis or set desired size for the output image.
void cv::resize (InputArray src, OutputArray dst, Size dsize, double fx = 0, double fy = 0, int interpolation = INTER_LINEAR)
parameter | required/optional | description |
---|---|---|
src | required | Source image |
dst | required | Destination image |
dsize | required | Desired size for the destination image |
fx(=0) | optional | Horizontal axis scale |
fy(=0) | optional | Vertical axis scale |
interpolation (=INTER_LINEAR) |
optional | One of the interpolation algorithm methods (see InterpolationFlags in OpenCV document) |
If the following conditions apply, OpenCVA execute resizing process using DRP.
parameter | range/values | note |
---|---|---|
src | width: 2 - 3840 height: 2 - 2160 channels: 1 - 4 bit-depth: 8bit |
|
dsize | width: 2 - 3840 height: 2 - 2160 |
’0’ can also be set. In that case, “fx” and “fy” parameter are valid. |
fx | (src_img width * fx): 2 - 3840 | Valid if dsize is 0. |
fy | (src_img height * fy): 2 - 2160 | Valid if dsize is 0. |
interpolation | INTER_LINEAR |
Converts an input image from one color space to another.
void cv::cvtColor (InputArray src, OutputArray dst, int code, int dstCn = 0 )
parameter | required/optional | description |
---|---|---|
src | required | Source image |
dst | required | Destination image |
code | required | Color space conversion code (see ColorConversionCode in OpenCV document) |
dstCn (=0) | optional | Number of channels in the destination image. If the default parameter (=0) specified, the number of the channels is derived automatically from src and code |
If the following conditions apply, OpenCVA execute cvtColor process using DRP.
parameter | range/values | note |
---|---|---|
src | width: 4 or more, even height: 6 or more, even channels: 2 bit-depth: 8bit |
|
code | COLOR_YUV2RGB_UYVY COLOR_YUV2BGR_UYVY COLOR_YUV2RGB_YUY2 COLOR_YUV2BGR_YUY2 COLOR_YUV2RGB_YVYU COLOR_YUV2BGR_YVYU |
|
dstCn | 0 | Default value |
Converts an image from one color space to another where the source image is stored in two planes.
void cv::cvtColorTwoPlane (InputArray src1, InputArray src2, OutputArray dst, int code)
parameter | required/optional | description |
---|---|---|
src1 | required | Source image of the Y plane |
src2 | required | Source image of the U/V plane |
dst | required | Destination image |
code | required | Specifies the type of conversion |
If the following conditions apply, OpenCVA execute cvtColorTwoPlane process using DRP.
parameter | range/values | note |
---|---|---|
src1 | width: 4 or more, even height: 6 or more, even channels: 1 bit-depth: 8bit |
|
src2 | width: Half size of src1 height: Half size of src1 channels: 2 bit-depth: 8bit |
|
code | COLOR_YUV2BGR_NV12 COLOR_YUV2RGB_NV12 COLOR_YUV2BGR_NV21 COLOR_YUV2RGB_NV21 |
GaussianBlur is one of image blurring (image smoothing) function. In this function a Gaussian kernel is used to produce the smoothed image.
GaussianBlur process with DRP supports following Gaussian filters with kernel sizes.
Kernel size: 3, 5, or 7
Filters used are as follows.
void GaussianBlur (InputArray src, OutputArray dst, Size ksize, double sigmaX, double sigmaY = 0, int borderType = BORDER_DEFAULT)
parameter | required/optional | description |
---|---|---|
src | required | Source image |
dst | required | Destination image |
ksize | required | Gaussian kernel size |
sigmaX | required | Gaussian kernel standard deviation in X direction |
sigmaY (=0) | optional | Gaussian kernel standard deviation in Y direction |
borderType (=BORDER_DEFAULT) |
optional | Pixel extrapolation method (see BorderTypes in OpenCV document) |
If the following conditions and apply, OpenCVA execute GaussianBlur process using DRP.
parameter | range/values | note |
---|---|---|
src | width: 16 or more, even height: 16 or more, even channels: 1 - 4 bit-depth: 8bit size limit: 4K(3840x2160) |
|
ksize | {3,3}, {5,5}, or {7,7} | Gaussian kernel size = 3, 5, or 7 |
sigmaX | 0* | |
sigmaY | 0* | Default value |
borderType | BORDER_DEFAULT | Default value |
*:In addition to above table, DRP is used in parameter combination “src channels = 1, ksize = {7,7}, sigmaX = 2, sigmaY = 2”.
Expands the area of bright areas in the image.
void cv::dilate (InputArray src, OutputArray dst, InputArray kernel, Point anchor = Point (-1, -1), int iterations = 1, int borderType = BORDER_CONSTANT, const Scalar & borderValue = morphologyDefaultBorderValue() )
parameter | required/optional | description |
---|---|---|
src | required | Input image |
dst | required | Destination image |
kernel | required | Dilate kernel size |
anchor (=Point (-1, -1) ) |
optional | Position of the anchor within the element. Default value (-1, -1) means that the anchor is at the element center. |
iterations (=1) | optional | Number of times dilation |
borderType (=BORDER_CONSTANT) |
optional | Pixel extrapolation method |
borderValue (=morphologyDefaultBorderValue()) |
optional | Border value (see morphologyDefaultBorderValue() in OpenCV document) |
If the following conditions apply, OpenCVA execute dilate process using DRP.
parameter | range/values | note |
---|---|---|
src | width: 16 or more, even height: 16 or more, even channels: 1 - 4 bit-depth: 8bit size limit: 4K(3840x2160) |
|
kernel | Mat() | 3 x 3 rectangular structuring element |
anchor | (-1, -1) | Default value |
iterations | 1 - 3840 | |
borderType | BORDER_CONSTANT or BORDER_REPLICATE |
|
borderValue | morphologyDefaultBorderValue() | Default value |
Reduce the area of bright areas in the image.
void cv::erode (InputArray src, OutputArray dst, InputArray kernel, Point anchor = Point (-1, -1), int iterations = 1, int borderType = BORDER_CONSTANT, const Scalar & borderValue = morphologyDefaultBorderValue() )
parameter | required/optional | description |
---|---|---|
src | required | Input image |
dst | required | Destination image |
kernel | required | Erode kernel size |
anchor (=Point (-1, -1) ) |
optional | Position of the anchor within the element. Default value (-1, -1) means that the anchor is at the element center. |
iterations (=1) | optional | Number of times dilation |
borderType (=BORDER_CONSTANT) |
optional | Pixel extrapolation method |
borderValue (=morphologyDefaultBorderValue()) |
optional | Border value (see morphologyDefaultBorderValue() in OpenCV document) |
If the following conditions apply, OpenCVA execute erode process using DRP.
parameter | range/values | note |
---|---|---|
src | width: 16 or more, even height: 16 or more, even channels: 1 - 4 bit-depth: 8bit size limit: 4K(3840x2160) |
|
kernel | Mat() | 3 x 3 rectangular structuring element |
anchor | (-1, -1) | Default value |
iterations | 1 - 3840 | |
borderType | BORDER_CONSTANT or BORDER_REPLICATE |
|
borderValue | morphologyDefaultBorderValue() | Default value |
Performs advanced morphological transformations.
This function uses both “dilate” and “erode” internally.
void cv::morphologyEX (InputArray src, OutputArray dst, int op, InputArray kernel, Point anchor = Point (-1, -1), int iterations = 1, int borderType = BORDER_CONSTANT, const Scalar & borderValue = morphologyDefaultBorderValue() )
parameter | required/optional | description |
---|---|---|
src | required | Input image |
dst | required | Destination image |
op | required | Type of a morphological operation (see MorphTypes in OpenCV document) |
kernel | required | Kernel size |
anchor (=Point (-1, -1) ) |
optional | Position of the anchor within the element. Default value (-1, -1) means that the anchor is at the element center. |
iterations (=1) | optional | Number of times dilation |
borderType (=BORDER_CONSTANT) |
optional | Pixel extrapolation method |
borderValue (=morphologyDefaultBorderValue()) |
optional | Border value (see morphologyDefaultBorderValue() in OpenCV document) |
If the following conditions apply, OpenCVA execute morphologyEX process using DRP.
parameter | range/values | note |
---|---|---|
src | width: 16 or more, even height: 16 or more, even channels: 1 - 4 bit-depth: 8bit size limit: 4K(3840x2160) |
|
kernel | Mat() | 3 x 3 rectangular structuring element |
anchor | (-1, -1) | Default value |
iterations | 1 - 3840 | |
borderType | BORDER_CONSTANT or BORDER_REPLICATE |
|
borderValue | morphologyDefaultBorderValue() | Default value |
Convolves an image with the kernel.
void cv::filter2D (InputArray src, OutputArray dst, int ddepth, InputArray kernel, Point anchor = Point(-1,-1) , double delta = 0, int borderType = BORDER_DEFAULT )
parameter | required/optional | description |
---|---|---|
src | required | Source image |
dst | required | Destination image |
ddepth | required | Destination image depth |
kernel | required | Convolution kernel |
anchor (=Point(-1,-1) ) |
optional | Position of the anchor within the element. Default value (-1, -1) means that the anchor is at the element center. |
delta (=0) | optional | Value to add to destination image |
borderType (=BORDER_DEFAULT) |
optional | Pixel extrapolation method |
If the following conditions apply, OpenCVA execute filter2D process using DRP.
parameter | range/values | note |
---|---|---|
src | width: 16 or more, even height: 16 or more, even channels: 1 - 4 bit-depth: 8bit size limit: 4K(3840x2160) |
|
ddepth | -1 | The same depth as the input image. |
kernel | size: {3,3} depth: CV_32F value: all element values are between -128 and 127 |
|
anchor | (-1, -1) | Default value |
delta | 0 | Default value |
borderType | BORDER_DEFAULT | Default value |
Calculates the first image derivatives.
void cv::Sobel (InputArray src, OutputArray dst, int ddepth, int dx, int dy, int ksize = 3, double scale = 1, double delta = 0, int borderType = BORDER_DEFAULT)
parameter | required/optional | description |
---|---|---|
src | required | Input image |
dst | required | Destination image |
ddepth | required | Destination image depth |
dx | required | Order of the derivative x |
dy | required | Order of the derivative y |
ksize (=3) | optional | Size of the extended Sobel kernel |
scale (=1) | optional | Scale factor |
delta (=0) | optional | Value to add to destination image |
intborderType (=BORDER_DEFAULT) |
optional Pixel extrapolation method |
If the following conditions apply, OpenCVA execute sobel process using DRP.
parameter | range/values | note |
---|---|---|
src | width: 16 or more, even height: 16 or more, even channels: 1 - 4 bit-depth: 8bit size limit: 4K(3840x2160) |
|
ddepth | CV_8U | |
dx, dy | “dx = 0, dy = 1” or “dx = 1, dy = 0” |
|
ksize | 3 | default value |
scale | 1 | default value |
delta | 0 | default value |
borderType | BORDER_DEFAULT | default value |
Transforms a grayscale image to a binary image according to the formulae.
void cv::adaptiveThreshold (InputArray src, OutputArray dst, double maxValue, int adaptiveMethod, int thresholdType, int blockSize, double C)
parameter | required/optional | description |
---|---|---|
src | required | Source image |
dst | required | Destination image |
maxValue | required | Non-zero value assigned to the pixels for which the condition is satisfied |
adaptiveMethod | required | Adaptive thresholding algorithm to use (see adaptiveThresholdTypes in OpenCV document) |
thresholdType | required | Threshold type (see ThresholdTypes in OpenCV document) |
blocksize | required | Size of a pixel neighborhood that is used to calculate a threshold value |
C | required | Constant subtracted from the mean |
If the following conditions apply, OpenCVA execute adaptiveThreshold process using DRP.
parameter | range/values | note |
---|---|---|
src | width: 16 or more, even height: 16 or more, even channels: 1 bit-depth: 8bit size limit: 4K(3840x2160) |
|
maxValue | 0 - 255 | |
adaptiveMethod | ADAPTIVE_THRES_MEAN_C | |
thresholdType | THRES_BINARY or THRES_BINARY_INV |
|
blocksize | 3 - 255, odd | |
C | 0 - 255 |
Compares a template against overlapped image regions.
Caution: “matchTemplate” function is very slow in some cases using DRP, depending on the parameters. Please test if the matchTemplate process with DRP performance is enough, using the assumed parameter.
void cv::matchTemplate(InputArray image, InputArray templ, OutputArray result, int method, InputArray mask = noArray() )
parameter | required/optional | description |
---|---|---|
image | required | Input image |
templ | required | Template image |
result | required | Map of comparison results Depth:32FC1 Width: image width - templ width +1 Height: image height - templ height +1 |
method | required | Specifying the comparison method (see TemplateMatchModes in OpenCV document) |
mask (= noArray() ) | optional | Mask of searched template |
If the following conditions apply, OpenCVA execute matchTemplate process using DRP.
parameter | range/values | note |
---|---|---|
src | width: 16 or more, even height: 16 or more, even channels: 1 - 4 bit-depth: 8bit size limit: FHD(1920x1080) |
|
templ | width: 8 or more, even height: 8 or more, even channels: same value as image bit-depth: same value as image size limit: image size |
|
method | CV_TM_SQDIFF CV_TM_SQDIFF_NORMED CV_TM_CCORR CV_TM_CCRR_NORMED |
|
mask | noArray() | Default value |
An affine transformation to an image.
void cv::warpAffine (InputArray src, OutputArray dst, InputArray M, Size dsize, int flags = INTER_LINEAR, int borderMode = BORDER_CONSTARNT, const Scalar & borderValue = Scalar() )
parameter | required/optional | description |
---|---|---|
src | required | Input image |
dst | required | Destination image |
M | required | Transformation matrix (2x3) |
dsize | required | Size of destination image |
flags (=INTER_LINEAR) |
optional | One of the interpolation algorithm methods (see InterpolationFlags in OpenCV document) |
borderMode (=BORDER_CONSTARNT) |
optional | Pixel extrapolation method (see BorderTypes in OpenCV document) |
borderValue (=Scalar() ) |
optional | Border value |
If the following conditions apply, OpenCVA execute warpAffine process using DRP.
parameter | range/values | note |
---|---|---|
src | width: 16 or more, even height: 16 or more, even channels: 1 - 4 bit-depth: 8bit size limit: 4K(3840x2160) |
|
M | CV_32F | |
dsize | width: 16 or more, even height: 16 or more, even size limit: 4K(3840x2160) |
|
flags | INTER_LINEAR | default value |
borderMode | BORDER_CONSTARNT | default value |
A perspective transformation to an image.
void cv::warpPerspective (InputArray src, OutputArray dst, InputArray M, Size dsize, int flags = INTER_LINEAR, int borderMode = BORDER_CONSTARNT, const Scalar & borderValue = Scalar() )
parameter | required/optional | description |
---|---|---|
src | required | Input image |
dst | required | Destination image |
M | required | Transformation matrix (3x3) |
dsize | required | Size of destination image |
flags (=INTER_LINEAR) |
optional | One of the interpolation algorithm methods (see InterpolationFlags in OpenCV document) |
borderMode (=BORDER_CONSTARNT) |
optional | Pixel extrapolation method (see BorderTypes in OpenCV document) |
borderValue (=Scalar() ) |
optional | Border value |
If the following conditions apply, OpenCVA execute warpPerspective process using DRP.
parameter | range/values | note |
---|---|---|
src | width: 16 or more, even height: 16 or more, even channels: 1 - 4 bit-depth: 8bit size limit: 4K(3840x2160) |
|
M | CV_32F | |
dsize | width: 16 or more, even height: 16 or more, even size limit: 4K(3840x2160) |
|
flags | INTER_LINEAR | default value |
borderMode | BORDER_CONSTARNT | default value |
Blurs an image and downsamples it.
void cv::pyrDown (InputArray src, OutputArray dst, const Size & dstsize = Size(), int borderType = BORDER_DEFAULT)
parameter | required/optional | description |
---|---|---|
src | required | Input image |
dst | required | Destination image |
dstsize(=Size()) | optional | Destination image size |
intborderType (=BORDER_DEFAULT) |
optional | Pixel extrapolation method |
If the following conditions apply, OpenCVA execute pyrDown process using DRP.
parameter | range/values | note |
---|---|---|
src | width: 16 or more, even height: 16 or more, even channels: 1 - 4 bit-depth: 8bit size limit: 4K(3840x2160) |
|
dstsize | Size() or Size( width/2, height/2) |
|
borderType | BORDER_DEFAULT | default value |
Blurs an image and upsamples it.
void cv::pyrUp (InputArray src, OutputArray dst, const Size & dstsize = Size(), int borderType = BORDER_DEFAULT)
parameter | required/optional | description |
---|---|---|
src | required | Source image |
dst | required | Destination image |
dstsize(=Size()) | optional | Destination image size |
intborderType (=BORDER_DEFAULT) |
optional | Pixel extrapolation method |
If the following conditions apply, OpenCVA execute pyrUp process using DRP.
parameter | range/values | note |
---|---|---|
src | width: 8 or more, even height: 8 or more, even channels: 1 - 4 bit-depth: 8bit size limit: FHD(1920x1080) |
|
dstsize | Size() or Size( width*2, height*2) |
|
borderType | BORDER_DEFAULT | default value |
Detects corners using the FAST algorithm.
void cv::Fast (InputArray image, std::vector<KeyPoint> &keypoints, int threshold, bool nonmaxSuppression = true, FastFeatureDetector::DetectorType type)
parameter | required/optional | description |
---|---|---|
image | required | Input image (grayscale image) |
keypoints | required | keypoints detected on the image |
threshold | required | threshold on difference between intensity of the central pixel and pixels of a circle around this pixel. |
nonmaxSuppression (=true) | optional | if true, non-maximum suppression is applied to detected corners (keypoints). |
type (=FastFeatureDetector::TYPE_9_16) |
optional | detector type |
If the following conditions apply, OpenCVA execute FAST process using DRP.
parameter | range/values | note |
---|---|---|
image | width: 16 or more, even height: 16 or more, even channels: 1 bit-depth: 8bit size limit: 4K(3840x2160) |
|
threshold | 0 - 255 | |
nonmaxSuppression | true | default value |
type | FastFeatureDetector::TYPE_9_16 | default value |
This chapter describes API functions to control OpenCVA.
Disable or enable DRP used for OpenCV process.
OCA_Activate
int OCA_Activate (unsigned long* OCA_list);
OCA_list : OpenCVA function activation table.
0 : OK
-1: Error
Disable or enable DRP used for OpenCV, for each of OpenCV function. By default, all are enabled. If disabled, the OpenCV function execute by CPU.
The argument of the API function is the array variable OCA_list[]. See the following table for OCA_list[] index and OpenCV functions.
index of OCA_list[] | OpenCV function |
---|---|
0 | resize |
2 | cvtColor, cvtColorTwoPlane |
4 | GaussianBlur |
5 | dilate, morphologyEX |
6 | erode, morphologyEX |
7 | Filter2D |
8 | Sobel |
9 | adaptiveThreshold |
10 | matchTemplate |
11 | warpAffine |
12 | pyrDown |
13 | perUp |
14 | warpPerspective |
15 | FAST |
Others | (unused) |
note: OCA_list[] table type and size is “unsigned long OCA_list[16]”.
Setting the OCA_list[index] to 1 and then executing OCA_Activate(), then the corresponding DRP is enabled. If 0 is set, it is desabled.
Values other than 0 and 1 are ignored.
unsigned long OCA_list[16];
for(int i=0; i<16; i++)OCA_list[i]=2;
/* Disable DRP(Sobel) */
OCA_list[8] = 0; //Disable
OCA_Activate( &OCA_list[0] );
Sets the behavior when DRP conflicts occur (exception error occurs or not).
OCA_ConflictNotification
void OCA_ConflictNotification (int oca_conflict );
oca_conflict 0: When DRP conflicts, exception error occurs.
Not 0: When DRP conflicts, exception error does not occur, and the OpenCV function is executed by CPU.
Sets the behavior when DRP conflicts occur (exception error occurs or not). By default, it causes an exception error do to DRP conflict.
e.g., execute OpenCV using DRP while the MPEG4 Encode process is using the DRP.
The error code for exception errors is -501.
The following is a sample to handle exception error.
/* Exception error enable */
OCA_ConflictNotification( 0 );
try{
cv::GaussianBlur(src, dst, {7,7},0,0);
} catch(cv::Exception& e) {
if( e == -501 ){/* exception error handling */}
}
This chapter describes DRP conflict.
OpenCVA use “Dynamically Reconfigurable Processor” (=DRP). And the MPEG4 encoding and decoding functions use the same DRP. There is only one DRP used by these functions on a device. And the function occupies the DRP while it is executing.
If the OpenCV use the DRP but is unable to do so because the DRP is already being used by another function, this is called “DRP conflict.”
OpenCVA cause DRP conflicts in the following 2cases:
case1. Case of start OpenCV using DRP, while the MPEG4 encode/decode function uses DRP in parallel.
case2. Case of start OpenCV using DRP, while other OpenCV is using DRP in another process (or thread).
If neither of the 2cases matches, then the DRP does not conflict. i.e.,
- MPEG4 encoding/decoding and OpenCV using DRP are executed sequentially.
- OpenCV using DRP is executed by single process/thread.
If the OpenCVA occurred DRP conflict, it raises an exception error or execute OpenCV function by CPU instead of the DRP. (selected by the OCA_ConflictNotification()).
If the DRP conflict occur exception error, the user application must be able to handle the exception error, as in Chapter 5.2 sample.
If the OpenCV function executed by CPU, the user applications do not need to support anything, but performance of the OpenCV should be confirmed.