❓ [Question] Unknown type name '__torch__.torch.classes.tensorrt.Engine'

[yokosyun]

❓ Question

when I tried to load trt module which saved with python

torch::jit::load(trtorch_path);

I got this error

terminate called after throwing an instance of 'torch::jit::ErrorReport'
  what():  
Unknown type name '__torch__.torch.classes.tensorrt.Engine':
Serialized   File "code/__torch__/models/backbone.py", line 4
  __parameters__ = []
  __buffers__ = []
  __torch___models_backbone_Backbone_trt_engine_ : __torch__.torch.classes.tensorrt.Engine
                                                   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
  def forward(self_1: __torch__.models.backbone.Backbone_trt,
    input_0: Tensor) -> Tensor:

I linked libtrtorch.so as bellow

target_link_libraries(
    ${ProjectTargetLibName}
    PRIVATE
        ${TORCH_LIBRARIES}
        /opt/trtorch/lib/libtrtorch.so
)

maybe I need to compile trt_module in c++ instead of python??

What you have already tried

I can save trt_module and load and run with python without any problem

Environment

• PyTorch 1.9
• Libtorch 1.9
• OS (Linux):
• CUDA 11.0:
• TensorRT 8.0
• TRTorch 0.4(libtrtorch-v0.4.0-cudnn8.2-tensorrt8.0-cuda11.1-libtorch-1.9.0.tar.gz)

[narendasan]

This error occurs when you don't load the trtorch runtime. Make sure the libraries are properly linked in your program. One thing that is pretty common on the inference side if you aren't doing compilation is because you don't directly use any APIs from trtorch, compilers may silently ignore linking the library. Try linking the library with -Wl,--no-as-needed. Also if you aren't compiling the PyTorch model in C++ all you need to link is libtrtorchrt.so which just had the runtime parts. It is a much smaller dependency than the full compiler library

[yokosyun]

thanks!
it works by adding -Wl,--no-as-needed

cmake_minimum_required(VERSION 3.12 FATAL_ERROR)

project(trtorchrt_example)

include_directories(
    /opt/libtorch-1.9/include
    /opt/trtorch/include/
    )

add_executable(${PROJECT_NAME} main.cpp)
target_link_libraries (${PROJECT_NAME} -Wl,--no-as-needed /opt/trtorch/lib/libtrtorchrt.so)
target_link_libraries (${PROJECT_NAME} 
                                       /opt/libtorch-1.9/lib/libtorch.so
                                       /opt/libtorch-1.9/lib/libtorch_cuda.so
                                       /opt/libtorch-1.9/lib/libtorch_cpu.so
                                       /opt/libtorch-1.9/lib/libtorch_global_deps.so
                                       /opt/libtorch-1.9/lib/libbackend_with_compiler.so
                                       /opt/libtorch-1.9/lib/libc10.so
                                       /opt/libtorch-1.9/lib/libc10_cuda.so
                                        )

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/trtorch/lib