Home > Mobile >  JETSON ORIN: setUpNet DNN module was not built with CUDA backend; switching to CPU
JETSON ORIN: setUpNet DNN module was not built with CUDA backend; switching to CPU

Time:11-29

I am working on a NVIDIA Jetson Orin to enable OpenCV with CUDA – DNN support.

My system specifications: JETSON ORIN Ubuntu 20.04 CUDA 11.4 CUDNN 8.6.0 OpenCV 4.5.4 YOLOv3

I am running as root on my Jetson Orin (Linux) or running it remotely from my Visual Studio 2019 using this code https://learnopencv.com/deep-learning-based-object-detection-using-yolov3-with-opencv-python-c/. It works with the object detection inference. But the problem is, it is switching back to CPU, not using the CUDA-GPU support. I did compile the OPENCV configuration with CUDA, CUDNN and OPENCV_DNN_CUDA and BUILD_OPENCV_DNN and other modules.

My cmake command line:

…:~$ cd tk_ws/opencv-4.5.4/build/

…/tk_ws/opencv-4.5.4/build$ cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local -D  WITH_TBB=ON -D  ENABLE_FAST_MATH=ON -D  CUDA_FAST_MATH=ON -D  WITH_CUBLAS=ON -D  WITH_CUDA=ON -D  CUDA_ARCH_BIN=8.7 -D  BUILD_opencv_cudacodec=OFF -D  WITH_CUDNN=ON -D  OPENCV_DNN_CUDA=ON -D  WITH_V4L=ON -D  WITH_QT=OFF
-D  WITH_OPENGL=ON -D  WITH_GSTREAMER=ON -D  OPENCV_GENERATE_PKGCONFIG=ON -D OPENCV_PC_FILE_NAME=opencv4.pc -D  OPENCV_ENABLE_NONFREE=ON -D OPENCV_PYTHON3_INSTALL_PATH=/usr/lib/python3.8/dist-packages -D OPENCV_EXTRA_MODULES_PATH=/home/…/tk_ws/opencv_contrib-4.5.4/modules
-D  INSTALL_PYTHON_EXAMPLES=OFF -D  INSTALL_C_EXAMPLES=OFF -D  BUILD_EXAMPLES=OFF -D  BUILD_OPENCV_DNN=ON -D  BUILD_OPENCV_WORLD=ON
-D CUDNN_VERSION=8.6.0 -D CUDNN_LIBRARY=/usr/lib/aarch64-linux-gnu/libcudnn.so -D CUDNN_INCLUDE_DIR=/usr/include -D PYTHON3_EXECUTABLE=/usr/bin/python3
-D PYTHON3_INCLUDE_DIR=/usr/include/python3.8 -D PYTHON3_LIBRARY=/usr/lib/aarch64-linux-gnu/libpython3.8.so -D PYTHON3_NUMPY_INCLUDE_DIRS=/usr/lib/python3/dist-packages/numpy/core/include
-D  PYTHON3_PACKAGES_PATH=/usr/lib/python3/dist-packages -D OpenCV_DIR=/home/…/tk_ws/opencv-4.5.4/build ..

In my configuration,

**1. General configuration for OpenCV 4.5.4 Version control: unknown

NVIDIA CUDA: YES (ver 11.4, CUFFT CUBLAS FAST_MATH)

NVIDIA GPU arch: 87

NVIDIA PTX archs:

cuDNN: YES (ver 8.6.0)**

and the

net.setPreferableBackend(DNN_BACKEND_OPENCV); net.setPreferableTarget(DNN_TARGET_CPU);

throw the error:

setUpNet DNN module was not built with CUDA backend; switching to CPU


Also, the test 1 code (test.cpp) test.cpp

     $  g   test.cpp `pkg-config opencv4 --cflags --libs` -o test
     $ ./test
     (opencv_dnn_cuda) ………:$ cd workspace/opencv-4.5.4
     (opencv_dnn_cuda) ……….:/workspace/opencv-4.5.4$ g   test.cpp pkg-config opencv4 –cflags –libs -o test
     (opencv_dnn_cuda) ………..:~/workspace/opencv-4.5.4$ ./test
    Error: OpenCV(4.5.4) /home/ubuntu/build_opencv/opencv/modules/core/include/opencv2/core/private.cuda.hpp:106: error: (-216:No CUDA support) The library is compiled without CUDA support in function ‘throw_no_cuda’
    
    0.000296

For another test 2 in python test.py

the result is as below: My results are

`in the virtual environment
lenovo: ~ /workspace/opencv-4.5.4$ cd build/ workon opencv_dnn_cuda
 (opencv_dnn_cuda) lenovo:~ /workspace/opencv-4.5.4/build$ python3 test.py
CUDA using GPU — 0.76786208152771 seconds —
CPU — 1.6167230606079102 seconds —`

`without virtual environment
lenovo:$ cd workspace/opencv-4.5.4/build
lenovo:/workspace/opencv-4.5.4/build$ python3 test.py
CUDA using GPU — 1.4443564414978027 seconds —
CPU — 3.2225732803344727 seconds —`

The weird part in the first two situations is: [ WARN:0] global /home/ubuntu/build_opencv/opencv/modules/dnn/src/dnn.cpp (1447) setUpNet DNN module was not built with CUDA backend; switching to CPU

As I mentioned above, the same weird issue repeats here too. I have no path /home/ubuntu/build_opencv/opencv/

Instead my actual path is /home/lenovo/workspace/opencv-4.5.4/modules/dnn/src/dnn.cpp


  1. $ python3

    Python 3.8.10 (default, Jun 22 2022, [20:18:18] [GCC 9.4.0] on linux Type “help”, “copyright”, “credits” or “license” for more information.

     import cv2
    
     cv2.version
    
     ‘4.5.4’
    
     exit()
    

pkg-config --modversion opencv4
4.5.4

Requesting for your kind help.

Thank you.

In addition,

As suggested by @Micka,

I have enabled CUDA_ARCH_PTX=“” and redid the make, sudo make install steps for opencv installation. Then tried a sample.cpp code for checking out the CUDA enabled test.

1. sample.cpp

#include <iostream>
#include "opencv2/opencv.hpp"
#include "opencv2/core/cuda.hpp"
#include "opencv2/cudaarithm.hpp"
#include "opencv2/cudaoptflow.hpp"
#include <opencv2/core/utility.hpp>
#include "opencv2/core.hpp"

int main (int argc, char* argv[])
{
    try
    {
        cv::cuda::GpuMat src_host = cv::imread("/home/...../test1.png", CV_LOAD_IMAGE_GRAYSCALE);
        cv::cuda::GpuMat dst, src;
        src.upload(src_host);

        cv::cuda::threshold(src, dst, 128.0, 255.0, CV_THRESH_BINARY);

        cv::cuda::GpuMat result_host;
        dst.download(result_host);

        cv::imshow("Result", result_host);
        cv::waitKey();
    }
    catch(const cv::Exception& ex)
    {
        std::cout << "Error: " << ex.what() << std::endl;
    }
    return 0;
}

Result:

g   sample.cpp `pkg-config opencv4 --cflags --libs` -o sample

sample.cpp: In function ‘int main(int, char**)’: sample.cpp:13:47: error: conversion from ‘cv::Mat’ to non-scalar type ‘cv::cuda::GpuMat’ requested 13 | cv::cuda::GpuMat src_host = cv::imread("/home/sesotec-ai-2/darknet/test1.png", cv::IMREAD_GRAYSCALE); | ~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ sample.cpp:17:53: error: ‘CV_THRESH_BINARY’ was not declared in this scope 17 | cv::cuda::threshold(src, dst, 128.0, 255.0, CV_THRESH_BINARY); | ^~~~~~~~~~~~~~~~

Test 2: test.cpp

#include <iostream>
#include <ctime>
#include <cmath>
#include "bits/time.h"

#include <opencv2/core.hpp>
#include <opencv2/opencv.hpp>
#include <opencv2/highgui.hpp>
#include <opencv2/imgproc.hpp>
#include <opencv2/imgcodecs.hpp>

#include <opencv2/core/cuda.hpp>
#include <opencv2/cudaarithm.hpp>
#include <opencv2/cudaimgproc.hpp>
#include "opencv2/core/cuda.hpp"
#include "opencv2/cudaarithm.hpp"
#include "opencv2/cudaoptflow.hpp"
#include <opencv2/core/utility.hpp>
#include "opencv2/core.hpp"
#include <opencv2/world.hpp>

#define TestCUDA true
using namespace cv;
using namespace cuda;

int main() {
    std::clock_t begin = std::clock();

        try {
            cv::String filename = "/home/.../.../test1.png";
            cv::Mat srcHost = cv::imread(filename, cv::IMREAD_GRAYSCALE);

            for(int i=0; i<1000; i  ) {
                if(TestCUDA) {
                    cv::cuda::GpuMat dst, src;
                    src.upload(srcHost);

                    //cv::cuda::threshold(src,dst,128.0,255.0, CV_THRESH_BINARY);
                    cv::cuda::bilateralFilter(src,dst,3,1,1);

                    cv::Mat resultHost;
                    dst.download(resultHost);
                } else {
                    cv::Mat dst;
                    cv::bilateralFilter(srcHost,dst,3,1,1);
                }
            }

            //cv::imshow("Result",resultHost);
            //cv::waitKey();

        } catch(const cv::Exception& ex) {
            std::cout << "Error: " << ex.what() << std::endl;
        }

    std::clock_t end = std::clock();
    std::cout << double(end-begin) / CLOCKS_PER_SEC  << std::endl;
}

Result:

 $ g   test.cpp `pkg-config opencv4 --cflags --libs` -o test  
 $ ./test

Error: OpenCV(4.5.4) /home/ubuntu/build_opencv/opencv/modules/core/include/opencv2/core/private.cuda.hpp:106: error: (-216:No CUDA support) The library is compiled without CUDA support in function 'throw_no_cuda'

0.047034

Build Information in the test.py

import cv2 as cv; 
print(cv.getBuildInformation())

                                    

> $ python3 test.py
> 
> CUDA using GPU --- 1.428098201751709 seconds --- CPU ---
> 3.2391397953033447 seconds ---
> 
> General configuration for OpenCV 4.5.4
> =====================================   Version control:               unknown
> 
>   Extra modules:
>     Location (extra):            /home/.../tk_ws/opencv_contrib-4.5.4/modules
>     Version control (extra):     unknown
> 
>   Platform:
>     Timestamp:                   2022-11-15T13:55:20Z
>     Host:                        Linux 5.10.65-tegra aarch64
>     CMake:                       3.16.3
>     CMake generator:             Unix Makefiles
>     CMake build tool:            /usr/bin/make
>     Configuration:               Release
> 
>   CPU/HW features:
>     Baseline:                    NEON FP16
> 
>   C/C  :
>     Built as dynamic libs?:      YES
>     C   standard:                11
>     C   Compiler:                /usr/bin/c    (ver 9.4.0)
>     C   flags (Release):         -fsigned-char -ffast-math -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections    -fvisibility=hidden -fvisibility-inlines-hidden -fopenmp -O3 -DNDEBUG  -DNDEBUG
>     C   flags (Debug):           -fsigned-char -ffast-math -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections    -fvisibility=hidden -fvisibility-inlines-hidden -fopenmp -g  -O0 -DDEBUG -D_DEBUG
>     C Compiler:                  /usr/bin/cc
>     C flags (Release):           -fsigned-char -ffast-math -W -Wall -Werror=return-type -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections    -fvisibility=hidden -fopenmp -O3 -DNDEBUG  -DNDEBUG
>     C flags (Debug):             -fsigned-char -ffast-math -W -Wall -Werror=return-type -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections    -fvisibility=hidden -fopenmp -g  -O0 -DDEBUG -D_DEBUG
>     Linker flags (Release):      -Wl,--gc-sections -Wl,--as-needed
>     Linker flags (Debug):        -Wl,--gc-sections -Wl,--as-needed
>     ccache:                      NO
>     Precompiled headers:         NO
>     Extra dependencies:          m pthread cudart_static dl rt nppc nppial nppicc nppidei nppif nppig nppim nppist nppisu nppitc npps
> cublas cudnn cufft -L/usr/local/cuda-11.4/lib64
> -L/usr/lib/aarch64-linux-gnu
>     3rdparty dependencies:
> 
>   OpenCV modules:
>     To be built:                 aruco barcode bgsegm bioinspired calib3d ccalib core cudaarithm cudabgsegm cudacodec cudafeatures2d
> cudafilters cudaimgproc cudalegacy cudaobjdetect cudaoptflow
> cudastereo cudawarping cudev datasets dnn dnn_objdetect dnn_superres
> dpm face features2d flann freetype fuzzy gapi hfs highgui img_hash
> imgcodecs imgproc intensity_transform line_descriptor mcc ml objdetect
> optflow phase_unwrapping photo plot python2 python3 quality rapid reg
> rgbd saliency shape stereo stitching structured_light superres
> surface_matching text tracking ts video videoio videostab
> wechat_qrcode world xfeatures2d ximgproc xobjdetect xphoto
>     Disabled:                    -
>     Disabled by dependency:      -
>     Unavailable:                 alphamat cvv hdf java julia matlab ovis sfm viz
>     Applications:                tests perf_tests examples apps
>     Documentation:               NO
>     Non-free algorithms:         YES
> 
>   GUI:
>     GTK :                        YES (ver 3.24.20)
>       GThread :                  YES (ver 2.64.6)
>       GtkGlExt:                  NO
>     VTK support:                 NO
> 
>   Media I/O:
>     ZLib:                        /usr/lib/aarch64-linux-gnu/libz.so (ver 1.2.11)
>     JPEG:                        /usr/lib/aarch64-linux-gnu/libjpeg.so (ver 80)
>     WEBP:                        build (ver encoder: 0x020f)
>     PNG:                         /usr/lib/aarch64-linux-gnu/libpng.so (ver 1.6.37)
>     TIFF:                        /usr/lib/aarch64-linux-gnu/libtiff.so (ver 42 / 4.1.0)
>     JPEG 2000:                   build (ver 2.4.0)
>     OpenEXR:                     /usr/lib/aarch64-linux-gnu/libImath.so
> /usr/lib/aarch64-linux-gnu/libIlmImf.so
> /usr/lib/aarch64-linux-gnu/libIex.so
> /usr/lib/aarch64-linux-gnu/libHalf.so
> /usr/lib/aarch64-linux-gnu/libIlmThread.so (ver 2_3)
>     HDR:                         YES
>     SUNRASTER:                   YES
>     PXM:                         YES
>     PFM:                         YES
> 
>   Video I/O:
>     DC1394:                      YES (2.2.5)
>     FFMPEG:                      YES
>       avcodec:                   YES (58.54.100)
>       avformat:                  YES (58.29.100)
>       avutil:                    YES (56.31.100)
>       swscale:                   YES (5.5.100)
>       avresample:                YES (4.0.0)
>     GStreamer:                   YES (1.16.3)
>     v4l/v4l2:                    YES (linux/videodev2.h)
> 
>   Parallel framework:            OpenMP
> 
>   Trace:                         YES (with Intel ITT)
> 
>   Other third-party libraries:
>     Lapack:                      NO
>     Eigen:                       NO
>     Custom HAL:                  YES (carotene (ver 0.0.1))
>     Protobuf:                    build (3.5.1)
> 
>   NVIDIA CUDA:                   YES (ver 11.4, CUFFT CUBLAS
> FAST_MATH)
>     NVIDIA GPU arch:             87
>     NVIDIA PTX archs:
> 
>   cuDNN:                         YES (ver 8.6.0)
> 
>   OpenCL:                        YES (no extra features)
>     Include path:                /home/../tk_ws/opencv-4.5.4/3rdparty/include/opencl/1.2
>     Link libraries:              Dynamic load
> 
>   Python 2:
>     Interpreter:                 /usr/bin/python2.7 (ver 2.7.18)
>     Libraries:                   /usr/lib/aarch64-linux-gnu/libpython2.7.so (ver 2.7.18)
>     numpy:                       /usr/lib/python2.7/dist-packages/numpy/core/include (ver 1.16.5)
>     install path:                lib/python2.7/dist-packages/cv2/python-2.7
> 
>   Python 3:
>     Interpreter:                 /usr/bin/python3 (ver 3.8.10)
>     Libraries:                   /usr/lib/aarch64-linux-gnu/libpython3.8.so (ver 3.8.10)
>     numpy:                       /usr/lib/python3/dist-packages/numpy/core/include (ver 1.17.4)
>     install path:                /usr/lib/python3/dist-packages/cv2/python-3.8
> 
>   Python (for build):            /usr/bin/python3
> 
>   Java:
>     ant:                         NO
>     JNI:                         NO
>     Java wrappers:               NO
>     Java tests:                  NO
> 
>   Install to:                    /usr/local
> -----------------------------------------------------------------

Device Info*

./deviceQuery Starting...

CUDA Device Query (Runtime API) version (CUDART static linking)

Detected 1 CUDA Capable device(s)

Device 0: "Orin"
CUDA Driver Version / Runtime Version          11.4 / 11.4
CUDA Capability Major/Minor version number:    8.7
Total amount of global memory:                 30623 MBytes (32110186496 bytes)
(016) Multiprocessors, (128) CUDA Cores/MP:    2048 CUDA Cores
GPU Max Clock rate:                            1300 MHz (1.30 GHz)
  Memory Clock rate:                             1300 Mhz
  Memory Bus Width:                              128-bit
  L2 Cache Size:                                 4194304 bytes
  Maximum Texture Dimension Size (x,y,z)         1D=(131072), 2D=(131072, 65536), 3D=(16384, 16384, 16384)
  Maximum Layered 1D Texture Size, (num) layers  1D=(32768), 2048 layers
  Maximum Layered 2D Texture Size, (num) layers  2D=(32768, 32768), 2048 layers
  Total amount of constant memory:               65536 bytes
  Total amount of shared memory per block:       49152 bytes
  Total shared memory per multiprocessor:        167936 bytes
  Total number of registers available per block: 65536
  Warp size:                                     32
  Maximum number of threads per multiprocessor:  1536
  Maximum number of threads per block:           1024
  Max dimension size of a thread block (x,y,z): (1024, 1024, 64)
  Max dimension size of a grid size    (x,y,z): (2147483647, 65535, 65535)
  Maximum memory pitch:                          2147483647 bytes
  Texture alignment:                             512 bytes
  Concurrent copy and kernel execution:          Yes with 2 copy engine(s)
  Run time limit on kernels:                     No
  Integrated GPU sharing Host Memory:            Yes
  Support host page-locked memory mapping:       Yes
  Alignment requirement for Surfaces:            Yes
  Device has ECC support:                        Disabled
  Device supports Unified Addressing (UVA):      Yes
  Device supports Managed Memory:                Yes
  Device supports Compute Preemption:            Yes
  Supports Cooperative Kernel Launch:            Yes
  Supports MultiDevice Co-op Kernel Launch:      Yes
  Device PCI Domain ID / Bus ID / location ID:   0 / 0 / 0
  Compute Mode:
     < Default (multiple host threads can use ::cudaSetDevice() with device simultaneously) >

deviceQuery, CUDA Driver = CUDART, CUDA Driver Version = 11.4, CUDA Runtime Version = 11.4, NumDevs = 1
Result = PASS

$ nvcc --version

nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2022 NVIDIA Corporation Built on Wed_May__4_00:02:26_PDT_2022 Cuda compilation tools, release 11.4, V11.4.239 Build cuda_11.4.r11.4/compiler.31294910_0

CodePudding user response:

It now worked.

After adding libopencv_world.so when the program is run helped.

Thank you all for the suggestions

  • Related