hown to compile model for multi inputs?

[shuaizzZ]

1）My model : out1, out2 = model(input1, input2)

2）How should i set compile settings, just like this:

trt_ts_module = torch_tensorrt.compile(torch_script_module,
inputs = [example_tensor, # Provide example tensor for input shape or...
torch_tensorrt.Input( # Specify input object with shape and dtype
min_shape=[1, 3, 224, 224],
opt_shape=[1, 3, 512, 512],
max_shape=[1, 3, 1024, 1024],
# For static size shape=[1, 3, 224, 224]
dtype=torch.half) # Datatype of input tensor. Allowed options torch.(float|half|int8|int32|bool)
],
enabled_precisions = {torch.half}, # Run with FP16)

[narendasan]

You would do something like:

trt_ts_module = torch_tensorrt.compile(model,
    inputs = [torch_tensorrt.Input(<INPUT_SHAPE_1>), torch_tensorrt.Input(<INPUT_SHAPE_2>),],
    enabled_precisions = {torch.half})

[shuaizzZ]

Thanks！！！What should I do when my inputs contain long tensor？

File "/data/web_server/model/model_pb/zs10_codes/python_grpc/utils/zs10_utils.py", line 470, in load_model_torch_tensorrt
model = torch_tensorrt.ts.compile(model, **compile_setting)
File "/usr/local/lib64/python3.6/site-packages/torch_tensorrt/ts/_compiler.py", line 116, in compile
compiled_cpp_mod = _C.compile_graph(module._c, _parse_compile_spec(spec))
RuntimeError: [Error thrown at core/util/trt_util.cpp:283] Expected type to be true but got false
Unsupported ATen data type Long

[shuaizzZ]

If my input size is fixed, should it be set like this?

torch_tensorrt.Input(
min_shape=[1024, 3, 224, 224],
opt_shape=[1024, 3, 224, 224],
max_shape=[1024, 3, 224, 224],
)

[narendasan]

If your input is fixed size then you only need to provide this once: torch_tensorrt.Input(shape=[1024, 3, 224, 224]) or just torch_tensorrt.Input([1024, 3, 224, 224])