Installing CUDA + cuDNN + TensorRT

SpireCV does not currently provide an official one-click installer for the vision environment on 40 Series GPUs. Adapting the environment for a 40 Series GPU therefore requires installing each component manually. The process is slightly more complex, but following this tutorial will help you complete it successfully.

1. Determine the CUDA Version to Install

Open a terminal and enter the following command:

nvidia-smi

After the graphics driver is installed correctly, information similar to the following will appear. Here, 535.183.01 is the graphics driver version. The CUDA version you install must be compatible with it.

Refer to the NVIDIA documentation to find the compatibility relationship between CUDA and graphics driver versions: CUDA and Graphics Driver Version Compatibility.

Based on the progression of versions in the table, CUDA 12.2 GA is recommended. GA means "General Availability," indicating that the release has passed internal testing and validation, reached a stable state, and is safe for all users to use in production. Version 12.2 is neither especially new nor old for 40 Series GPUs. For development, a stable version in the middle of the supported range is generally preferred because it has fewer bugs.

The official documentation should always be the final authority when selecting a version. As a general recommendation, choose a slightly older official release to avoid incompatibilities caused by other components not yet supporting the latest version.

2. Download and Install CUDA

Download the CUDA version selected above from the official CUDA Toolkit Archive.

Open the link and select your platform, as shown below.

Then, from your home directory, enter the official installation commands in the terminal in sequence:

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/cuda-ubuntu2004.pin

sudo mv cuda-ubuntu2004.pin /etc/apt/preferences.d/cuda-repository-pin-600

wget https://developer.download.nvidia.com/compute/cuda/12.2.0/local_installers/cuda-repo-ubuntu2004-12-2-local_12.2.0-535.54.03-1_amd64.deb

sudo dpkg -i cuda-repo-ubuntu2004-12-2-local_12.2.0-535.54.03-1_amd64.deb

sudo cp /var/cuda-repo-ubuntu2004-12-2-local/cuda-*-keyring.gpg /usr/share/keyrings/

sudo apt-get update

sudo apt-get -y install cuda

After installation, open the ~/.bashrc file in your home directory and add the following:

export PATH=/usr/local/cuda-12.2/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda-12.2/lib64:$LD_LIBRARY_PATH

This configures the environment variables for CUDA.
Then restart the computer.

root

3. Verify CUDA

After the computer starts, check whether CUDA was installed successfully by entering the following command in a terminal:

nvcc -V

If the following message appears:

Command 'nvcc' not found, but can be installed with:

sudo apt install nvidia-cuda-toolkit

Install the toolkit:

sudo apt install nvidia-cuda-toolkit

After installation, enter the command again:

nvcc -V

Under normal conditions, output similar to the following will indicate that CUDA version 12.2 is installed on the system.

4. About CUDA

CUDA (Compute Unified Device Architecture) is an integrated hardware and software technology developed by NVIDIA and is the company's official name for GPGPU computing. This technology allows users to use NVIDIA GPUs for computations beyond graphics processing and was the first development environment that enabled a GPU to be used with a C compiler. The CUDA Toolkit can compile only NVIDIA's own CUDA C code (it provides only linking capabilities for OpenCL)—that is, the portion that runs on the GPU—into PTX intermediate code or machine code for a specific NVIDIA GPU architecture (officially called "device code" by NVIDIA). The C/C++ code that runs on the CPU (officially called "host code" by NVIDIA) still relies on an external compiler, such as Microsoft Visual Studio on Microsoft Windows or primarily GCC on Linux.

The NVIDIA® CUDA® Toolkit provides a development environment for creating high-performance, GPU-accelerated applications. It enables you to develop, optimize, and deploy applications on GPU-accelerated embedded systems, desktop workstations, enterprise data centers, cloud-based platforms, and supercomputers. The toolkit includes GPU-accelerated libraries, debugging and optimization tools, a C/C++ compiler, and runtime libraries.

Its main functions include:

1. Accelerated computing: Uses the GPU's parallel-processing capabilities to accelerate scientific computing, machine learning, image processing, and other tasks. 2. Parallel processing: Supports large-scale parallel computing to improve task execution efficiency. 3. Development flexibility: Provides extensive libraries and tools that help developers implement complex algorithms on GPUs. 4. Cross-platform support: Supports multiple operating systems and programming languages, enabling broad use across different fields. 5. Deep-learning support: Provides optimized support for deep-learning frameworks such as TensorFlow and PyTorch to increase training speed.

CUDA significantly improves the performance of compute-intensive tasks, allowing complex computations to be completed in less time.