[CPU] Introduce FullyConnected, FCQuantized, FCCompressed, Placeholder

[EgorDuplensky]

Details:

1. Introduce the following operations to the internal opset

   • FullyConnected (MatMul with transposed constant second input)
   • FullyConnectedCompressed (FullyConnected with weights compression)
   • FullyConnectedQuantizedLegacy (FullyConnected with quantized activations and weights and dequantize scale and zero point pulled through the Op by LPT)
   • FullyConnectedQuantized (FullyConnected with quantization scales and zero points on activation, weights and outputs). Planned to be used in scope of dynamic quantization. Can be used for a static quantization as well in the future.
   • Unused inputs are presented as Constant input with Shape{0}

2. The following transformations were added / updated:

   • ConvertFullyConnectedToFullyConnectedCompressed (replaces proprietary FuseFCAndWeightsDecompression)
   • ConvertFCToFCQuantizedLegacy replaces proprietary FuseConvMatmulFCDeconvAndDQScales
   • FullyConnectedBiasFusion (added into CPU folder for now, needs to be checked and review by GPU team before adaptation to internal opset). Replaces proprietary FuseConvolutionMatMulDeconvAndBias
   • ConvertMatMulToFC updated to use ov::op::internal:FullyConnected, planned to be moved to internal opset after review from GPU team

Todo

• Clean up debug code
• Clean up extra cmake targets
• Perf regression check

Tickets:

• 149923

[EgorDuplensky]

There is an option to create an empty uninitialized Constant of undefined type -- just like that: make_shared<v0::Constant>(element::undefined, Shape{0}); I think we could reconsider the introduction of a Placeholder or simply inherit it from Constant if you need a separate entity.

This was my initial idea, since I also didn't like the PlaceHolder idea. And it didn't work, don't remember why. Let me try one more time :)

Placeholder replaced with an undefined Constant with an empty shape

[EgorDuplensky]

There is an option to create an empty uninitialized Constant of undefined type -- just like that: make_shared<v0::Constant>(element::undefined, Shape{0}); I think we could reconsider the introduction of a Placeholder or simply inherit it from Constant if you need a separate entity.

This was my initial idea, since I also didn't like the PlaceHolder idea. And it didn't work, don't remember why. Let me try one more time :)

Placeholder replaced with an undefined Constant with an empty shape

There is one 'issue' left
With this change we are making 'undefined' element type a valid element type.
It has an impact on serialization / deserialization. Such element type was not expected there before.
Also netron does not support it at the moment which is probably is not a big deal and can be fixed.
Also, now we basically do not have some 'invalid' element type inside enum.
Not sure if this is a real issue or not.
@jane-intel @praasz @itikhono What do you think?

[praasz]

There is an option to create an empty uninitialized Constant of undefined type -- just like that: make_shared<v0::Constant>(element::undefined, Shape{0}); I think we could reconsider the introduction of a Placeholder or simply inherit it from Constant if you need a separate entity.

This was my initial idea, since I also didn't like the PlaceHolder idea. And it didn't work, don't remember why. Let me try one more time :)

Placeholder replaced with an undefined Constant with an empty shape

There is one 'issue' left With this change we are making 'undefined' element type a valid element type. It has an impact on serialization / deserialization. Such element type was not expected there before. Also netron does not support it at the moment which is probably is not a big deal and can be fixed. Also, now we basically do not have some 'invalid' element type inside enum. Not sure if this is a real issue or not. @jane-intel @praasz @itikhono What do you think?

In serialization the undefined and dynamic types are converted to same string.
Maybe for this empty constant the dynamic can be used and only for empty const can be serialized as dynamic string (maybe can be applied as generic rule later).

The dynamic type is compatible with any other. The deserialization for type should work as it get ov type by name.

[EgorDuplensky]

There is an option to create an empty uninitialized Constant of undefined type -- just like that: make_shared<v0::Constant>(element::undefined, Shape{0}); I think we could reconsider the introduction of a Placeholder or simply inherit it from Constant if you need a separate entity.

This was my initial idea, since I also didn't like the PlaceHolder idea. And it didn't work, don't remember why. Let me try one more time :)

Placeholder replaced with an undefined Constant with an empty shape

There is one 'issue' left With this change we are making 'undefined' element type a valid element type. It has an impact on serialization / deserialization. Such element type was not expected there before. Also netron does not support it at the moment which is probably is not a big deal and can be fixed. Also, now we basically do not have some 'invalid' element type inside enum. Not sure if this is a real issue or not. @jane-intel @praasz @itikhono What do you think?

In serialization the undefined and dynamic types are converted to same string. Maybe for this empty constant the dynamic can be used and only for empty const can be serialized as dynamic string (maybe can be applied as generic rule later).

The dynamic type is compatible with any other. The deserialization for type should work as it get ov type by name.

ov::dynamic type is quite unique. It does not really play well with the backends we use.
Let's try to proceed with an undefined type for now. We can replace it with dynamic one later if necessary

[EgorDuplensky]

Performance regression checks passed. Ready for merge