官方tensorrt备注笔记
APACHE-2.0 License
https://www.yuque.com/huangzhongqing/gk5f7m/ysgfhl
onnx:sampleOnnxMNIST
pluginhttps://github.com/HuangCongQing/tensorrt-plugin
(PCL/ROS+DL) (PCL,ROS)+ApolloAutoware(ros2)BEVSLAM(+) ++&
This repository contains the Open Source Software (OSS) components of NVIDIA TensorRT. Included are the sources for TensorRT plugins and parsers (Caffe and ONNX), as well as sample applications demonstrating usage and capabilities of the TensorRT platform. These open source software components are a subset of the TensorRT General Availability (GA) release with some extensions and bug-fixes.
To build the TensorRT-OSS components, you will first need the following software packages.
TensorRT GA build
System Packages
Optional Packages
Containerized build
Toolchains and SDKs
PyPI packages (for demo applications/tests)
Code formatting tools (for contributors)
NOTE: onnx-tensorrt, cub, and protobuf packages are downloaded along with TensorRT OSS, and not required to be installed.
git clone -b master https://github.com/nvidia/TensorRT TensorRT
cd TensorRT
git submodule update --init --recursive
If using the TensorRT OSS build container, TensorRT libraries are preinstalled under /usr/lib/x86_64-linux-gnu
and you may skip this step.
Else download and extract the TensorRT GA build from NVIDIA Developer Zone.
Example: Ubuntu 18.04 on x86-64 with cuda-11.4
cd ~/Downloads
tar -xvzf TensorRT-8.2.3.0.Linux.x86_64-gnu.cuda-11.4.cudnn8.2.tar.gz
export TRT_LIBPATH=`pwd`/TensorRT-8.2.3.0
Example: Windows on x86-64 with cuda-11.4
cd ~\Downloads
Expand-Archive .\TensorRT-8.2.3.0.Windows10.x86_64.cuda-11.4.cudnn8.2.zip
$Env:TRT_LIBPATH = '$(Get-Location)\TensorRT-8.2.3.0'
$Env:PATH += 'C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\MSBuild\15.0\Bin\'
Linux Jetpack 4.6
), and click Continue.Download & Install Options
change the download folder and select Download now, Install later
. Agree to the license terms and click Continue.<TensorRT-OSS>/docker/jetpack_files
folder.For Linux platforms, we recommend that you generate a docker container for building TensorRT OSS as described below. For native builds, on Windows for example, please install the prerequisite System Packages.
The TensorRT-OSS build container can be generated using the supplied Dockerfiles and build script. The build container is configured for building TensorRT OSS out-of-the-box.
Example: Ubuntu 18.04 on x86-64 with cuda-11.4.2 (default)
./docker/build.sh --file docker/ubuntu-18.04.Dockerfile --tag tensorrt-ubuntu18.04-cuda11.4
Example: CentOS/RedHat 7 on x86-64 with cuda-10.2
./docker/build.sh --file docker/centos-7.Dockerfile --tag tensorrt-centos7-cuda10.2 --cuda 10.2
Example: Ubuntu 18.04 cross-compile for Jetson (aarch64) with cuda-10.2 (JetPack SDK)
./docker/build.sh --file docker/ubuntu-cross-aarch64.Dockerfile --tag tensorrt-jetpack-cuda10.2 --cuda 10.2
Example: Ubuntu 20.04 on aarch64 with cuda-11.4.2
./docker/build.sh --file docker/ubuntu-20.04-aarch64.Dockerfile --tag tensorrt-aarch64-ubuntu20.04-cuda11.4
Example: Ubuntu 18.04 build container
./docker/launch.sh --tag tensorrt-ubuntu18.04-cuda11.4 --gpus all
NOTE:
--tag
corresponding to build container generated in Step 1.sudo
password for Ubuntu build containers is 'nvidia'.--jupyter <port>
for launching Jupyter notebooks.Generate Makefiles or VS project (Windows) and build.
Example: Linux (x86-64) build with default cuda-11.4.2
cd $TRT_OSSPATH
mkdir -p build && cd build
cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DTRT_OUT_DIR=`pwd`/out
make -j$(nproc)
NOTE: On CentOS7, the default g++ version does not support C++14. For native builds (not using the CentOS7 build container), first install devtoolset-8 to obtain the updated g++ toolchain as follows:
yum -y install centos-release-scl
yum-config-manager --enable rhel-server-rhscl-7-rpms
yum -y install devtoolset-8
export PATH="/opt/rh/devtoolset-8/root/bin:${PATH}
Example: Linux (aarch64) build with default cuda-11.4.2
cd $TRT_OSSPATH
mkdir -p build && cd build
cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DTRT_OUT_DIR=`pwd`/out -DCMAKE_TOOLCHAIN_FILE=$TRT_OSSPATH/cmake/toolchains/cmake_aarch64-native.toolchain
make -j$(nproc)
Example: Native build on Jetson (aarch64) with cuda-10.2
cd $TRT_OSSPATH
mkdir -p build && cd build
cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DTRT_OUT_DIR=`pwd`/out -DTRT_PLATFORM_ID=aarch64 -DCUDA_VERSION=10.2
CC=/usr/bin/gcc make -j$(nproc)
NOTE: C compiler must be explicitly specified via
CC=
for nativeaarch64
builds of protobuf.
Example: Ubuntu 18.04 Cross-Compile for Jetson (aarch64) with cuda-10.2 (JetPack)
cd $TRT_OSSPATH
mkdir -p build && cd build
cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DTRT_OUT_DIR=`pwd`/out -DCMAKE_TOOLCHAIN_FILE=$TRT_OSSPATH/cmake/toolchains/cmake_aarch64.toolchain -DCUDA_VERSION=10.2 -DCUDNN_LIB=/pdk_files/cudnn/usr/lib/aarch64-linux-gnu/libcudnn.so -DCUBLAS_LIB=/usr/local/cuda-10.2/targets/aarch64-linux/lib/stubs/libcublas.so -DCUBLASLT_LIB=/usr/local/cuda-10.2/targets/aarch64-linux/lib/stubs/libcublasLt.so
make -j$(nproc)
NOTE: The latest JetPack SDK v4.6 only supports TensorRT 8.0.1.
Example: Windows (x86-64) build in Powershell
cd $Env:TRT_OSSPATH
mkdir -p build ; cd build
cmake .. -DTRT_LIB_DIR=$Env:TRT_LIBPATH -DTRT_OUT_DIR='$(Get-Location)\out' -DCMAKE_TOOLCHAIN_FILE=..\cmake\toolchains\cmake_x64_win.toolchain
msbuild ALL_BUILD.vcxproj
NOTE:
-DCUDA_VERSION=10.2
to the cmake command.ln -s $TRT_OUT_DIR/libnvinfer_plugin.so $TRT_OUT_DIR/libnvinfer_plugin.so.8
Required CMake build arguments are:
TRT_LIB_DIR
: Path to the TensorRT installation directory containing libraries.TRT_OUT_DIR
: Output directory where generated build artifacts will be copied.Optional CMake build arguments:
CMAKE_BUILD_TYPE
: Specify if binaries generated are for release or debug (contain debug symbols). Values consists of [Release
] | Debug
CUDA_VERISON
: The version of CUDA to target, for example [11.4.2
].CUDNN_VERSION
: The version of cuDNN to target, for example [8.2
].PROTOBUF_VERSION
: The version of Protobuf to use, for example [3.0.0
]. Note: Changing this will not configure CMake to use a system version of Protobuf, it will configure CMake to download and try building that version.CMAKE_TOOLCHAIN_FILE
: The path to a toolchain file for cross compilation.BUILD_PARSERS
: Specify if the parsers should be built, for example [ON
] | OFF
. If turned OFF, CMake will try to find precompiled versions of the parser libraries to use in compiling samples. First in ${TRT_LIB_DIR}
, then on the system. If the build type is Debug, then it will prefer debug builds of the libraries before release versions if available.BUILD_PLUGINS
: Specify if the plugins should be built, for example [ON
] | OFF
. If turned OFF, CMake will try to find a precompiled version of the plugin library to use in compiling samples. First in ${TRT_LIB_DIR}
, then on the system. If the build type is Debug, then it will prefer debug builds of the libraries before release versions if available.BUILD_SAMPLES
: Specify if the samples should be built, for example [ON
] | OFF
.GPU_ARCHS
: GPU (SM) architectures to target. By default we generate CUDA code for all major SMs. Specific SM versions can be specified here as a quoted space-separated list to reduce compilation time and binary size. Table of compute capabilities of NVIDIA GPUs can be found here. Examples:
-DGPU_ARCHS="80"
-DGPU_ARCHS="75"
-DGPU_ARCHS="70"
-DGPU_ARCHS="80 75"
TRT_PLATFORM_ID
: Bare-metal build (unlike containerized cross-compilation) on non Linux/x86 platforms must explicitly specify the target platform. Currently supported options: x86_64
(default), aarch64