不同个数的输出导致运行结果不一致

[Rudin6]

Description

（部分linear attetion代码）
hidden_states = query_KV / query_Z
return hidden_states, query_KV, query_Z

和

hidden_states = query_KV / query_Z
return hidden_states

上面两者方式onnx转tensorrt时，两者结果不一样，前者是正确的，后者会出现nan;
这相当于输出了中间状态会导致结果的正确性，该怎么解决这种问题哇？

后面测试：tensorrt 10.7会出现这种问题，10.6是正确的

[lix19937]

Maybe the fusion methods(tactic choice) are different.

[Rudin6]

How to solve it?

[lix19937]

You can compare the build logs of two onnx by trtexec --verbose --layerProfile.

[Rudin6]

Thanks! There is one other problem in Tensorrt 10.7.0 : when building the trt engine, we set the parameter value of config:
logger = trt.Logger(trt.Logger.VERBOSE)
config.profiling_verbosity = trt.ProfilingVerbosity.DETAILED
config.set_flag(trt.BuilderFlag.BF16)
config.set_flag(trt.BuilderFlag.TF32)
During the process of building the engine, only f32f32 gemm or tf32tf32 gemm was discovered, no bf16 gemm， resulting in bf16 is slower than fp16.
How to solve it ?