TensorRT

Important Classes of TensorRT

1. IBuilder

Builds an engine from a network definition. Do not inherit from this class, as doing so will break forward-compatibility of the API and ABI. click here for more details

Warning:
Do not inherit from this class, as doing so will break forward-compatibility of the API and ABI.

  1. INetworkDefinition: A network definition for input to the builder. A network definition defines the structure of the network, and combined with a IBuilderConfig, is built into an engine using an IBuilder. An INetworkDefinition can either have an implicit batch dimensions, specified at runtime, or all dimensions explicit, full dims mode, in the network definition. When a network has been created using createNetwork(), only implicit batch size mode is supported. The function hasImplicitBatchDimension() is used to query the mode of the network.
    A network with implicit batch dimensions returns the dimensions of a layer without the implicit dimension, and instead the batch is specified at execute/enqueue time. If the network has all dimensions specified, then the first dimension follows elementwise broadcast rules: if it is 1 for some inputs and is some value N for all other inputs, then the first dimension of each output is N, and the inputs with 1 for the first dimension are broadcast. Having divergent batch sizes across inputs to a layer is not supported.click here for more details

  2. IBuilderConfig: Holds properties for configuring a builder to produce an engine. click here for more details

2. ICudaEngine

An engine for executing inference on a built network, with functionally unsafe features. click here for more details

Warning:
Do not inherit from this class, as doing so will break forward-compatibility of the API and ABI.

  1. IRuntime: Allows a serialized functionally unsafe engine to be deserialized. click here for more details

Warning:
Do not inherit from this class, as doing so will break forward-compatibility of the API and ABI.

3. IExecutionContext

Context for executing inference using an engine, with functionally unsafe features.

Multiple execution contexts may exist for one ICudaEngine instance, allowing the same engine to be used for the execution of multiple batches simultaneously. If the engine supports dynamic shapes, each execution context in concurrent use must use a separate optimization profile. click here for more details

Warning
Do not inherit from this class, as doing so will break forward-compatibility of the API and ABI

阅读(59)
评论(0)
updated@2022-05-07
评论区
目录