Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TEST][DONT REVIEW] just enable kv cache fusion for hunyuan #28201

Draft
wants to merge 3 commits into
base: releases/2024/6
Choose a base branch
from

Conversation

zhaohb
Copy link
Contributor

@zhaohb zhaohb commented Dec 25, 2024

Details:

fix the issue that the hunyuan model cannot use openvino inference.

RuntimeError: Exception from src/inference/src/cpp/infer_request.cpp:245:
Exception from src/bindings/python/src/pyopenvino/core/infer_request.hpp:54:
Caught exception: Check 'TRShape::merge_into(output_shape, in_copy)' failed at src/core/shape_inference/include/concat_shape_inference.hpp:43:
While validating node 'opset1::Concat concat:__module.model.model.layers.0.self_attn/aten::cat/Concat_2 () -> ()' with friendly_name 'concat:__module.model.model.layers.0.self_attn/aten::cat/Concat_2':
Shape inference input shapes {[1,0,0,128],[1,10,2172,128]}
Argument shapes are inconsistent; they must have the same rank, and must have equal dimension everywhere except on the concatenation axis (axis 2).

Tickets:

  • ticket-id

@github-actions github-actions bot added category: GPU OpenVINO GPU plugin category: transformations OpenVINO Runtime library - Transformations labels Dec 25, 2024
@zhaohb zhaohb marked this pull request as ready for review December 25, 2024 08:50
@zhaohb zhaohb requested review from a team as code owners December 25, 2024 08:50
@zhaohb zhaohb requested review from itikhono and removed request for a team December 25, 2024 08:50
@zhaohb zhaohb marked this pull request as draft December 25, 2024 14:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
category: GPU OpenVINO GPU plugin category: transformations OpenVINO Runtime library - Transformations do_not_merge do_not_review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant