Segmentation fault when destroying ICudaEngine object with C++ API

[kostastsing]

Description

I have defined a class TRTEngine with the necessary runtime, engine and context as member variables as follows:

class TRTEngine {
public:
    ...
private: 
    nvinfer1::IRuntime* _runtime = nullptr;
    nvinfer1::ICudaEngine* _engine = nullptr;
    nvinfer1::IExecutionContext* _context = nullptr;

    cudaStream_t _stream = nullptr;
    ...
}

where these variables are initialized in the constructor as follows:

TRTEngine::TRTEngine(
    const std::string& engine_path) : _runtime(nvinfer1::createInferRuntime(trt_logger))
{
    initLibNvInferPlugins(_runtime->getLogger(), "");

    std::ifstream planfile(engine_path);
    std::stringstream plan_buffer;
    plan_buffer << planfile.rdbuf();
    std::string plan = plan_buffer.str();

    _engine = _runtime->deserializeCudaEngine(plan.data(), plan.size());

    if (!_engine) {
        Logger::instance().error("Cannot load TensorRT engine", __FILE__, __LINE__);
        return;
    }

    ...

    _context = _engine->createExecutionContext();
    if (!_context) {
        Logger::instance().error("Cannot create execution context", __FILE__, __LINE__);
        return;
    }

    ...

    if (cudaStreamCreate(&_stream)) {
        Logger::instance().error("Could not create cuda stream", __FILE__, __LINE__);
        return;
    }
}

and the destructor is defined as follows:

TRTEngine::~TRTEngine()
{
    cudaStreamSynchronize(_stream);
    cudaStreamDestroy(_stream);

    if (_context)
        delete _context;
    if (_engine)
        delete _engine;
    if (_runtime)
        delete _runtime;
}

The problem is that a segmentation fault is raised in the destructor when trying to delete _engine. I have also tried with smart pointers or reordering the declaration/deletion of the objects but the problem still remains. What may cause this?

Environment

TensorRT Version: 10.0

NVIDIA GPU: RTX 3060

NVIDIA Driver Version: 535.161.07

CUDA Version: 11.8

CUDNN Version: 8.9.6

Operating System: Ubuntu 22.04
Baremetal or Container (if so, version): nvidia/cuda:11.8.0-cudnn8-devel-ubuntu22.04 with TensorRT installed later

[zerollzeng]

Could you please provide a reproduce? Thanks!

[lix19937]

You can ref the source of https://github.com/NVIDIA/TensorRT/tree/release/10.0/samples/trtexec to modify your code.

[ttyio]

closing since no activity for more than 3 weeks, pls reopen if you still have question. thanks all!