Code

Dynamic KV cache allocation#1364

Code

Merged

ilya-lavrenov merged 39 commits intoopenvinotoolkit:masteropenvinotoolkit/openvino.genai:masterfrom popovaan:dynamic_kv_cache_allocationpopovaan/openvino.genai:dynamic_kv_cache_allocationCopy head branch name to clipboardDec 24, 2024

+480-112

Commits

Commits on Dec 11, 2024

• Dynamic KV-cache allocation.

  popovaan

  committed
  a8c290a

• Minor corrections.

  popovaan

  committed
  8b68108

Commits on Dec 12, 2024

• Fixed cpp tests, added tests of dynamic allocation.

  popovaan

  committed
  5c384a0

• Merge remote-tracking branch 'upstream/master' into dynamic_kv_cache_allocation

  popovaan

  committed
  4e97cf9

• Fixed typo.

  popovaan

  committed
  0f32f1f

• Test corrected.

  popovaan

  committed
  d3f15fa

• Minor corrections.

  popovaan

  committed
  ec7ca26

Commits on Dec 13, 2024

• Merge remote-tracking branch 'upstream/master' into dynamic_kv_cache_allocation

  popovaan

  committed
  543cbd6

• Minor corrections.

  popovaan

  committed
  175f241

• Minor correction.

  popovaan

  committed
  a6facc5

• Removed not used code.

  popovaan

  committed
  34f6d27

Commits on Dec 16, 2024

• Merge branch 'master' into dynamic_kv_cache_allocation

  popovaan

  authored
  0f50cb7

Commits on Dec 17, 2024

• Code optimizations.

  popovaan

  committed
  0c3bb28

• Code corrections.

  popovaan

  committed
  a105a9f

Commits on Dec 18, 2024

• Merge upsteam/master.

  popovaan

  committed
  7537997

Commits on Dec 19, 2024

• Added available memory check for GPU.

  popovaan

  committed
  a8531a5

• Minor correction.

  popovaan

  committed
  9043ba3

• Code corrections.

  popovaan

  committed
  9256f15

• Minor correction.

  popovaan

  committed
  d926303

• Used correct core instance.

  popovaan

  committed
  eb4d110

Commits on Dec 20, 2024

• Moved increasing of cache logic to scheduler.

  popovaan

  committed
  f94929c

• Merge upstream/master.

  popovaan

  committed
  a1e4973

• Made sheduler config not needed for prompt lookup.

  popovaan

  committed
  38a42d6

• Minor correction.

  popovaan

  committed
  c7d54dd

• Fixed error.

  popovaan

  committed
  51cb0a8

• Removed wrong changes.

  popovaan

  committed
  c4c8c25

• Fixed error.

  popovaan

  committed
  bfcf9ff

• Minor correction.

  popovaan

  committed
  11b5e33

• Removed wrong changes.

  popovaan

  committed
  64dab76

• Merge branch 'master' into dynamic_kv_cache_allocation

  ilya-lavrenov

  authored
  0d71053

Commits on Dec 23, 2024

• Fix.

  popovaan

  committed
  bb24a36

• Fix of cache increasing for gpu.

  popovaan

  committed
  13f9f08

Commits on Dec 24, 2024

• Merge branch 'master' into dynamic_kv_cache_allocation

  popovaan

  authored
  f04c06d

• Applied comments.

  popovaan

  committed
  eebac1f

• Update src/cpp/src/scheduler.hpp

  Show description for 2715110
  

  popovaan

  and

  ilya-lavrenov

  authored
  2715110

• Update src/cpp/src/scheduler.hpp

  Show description for 1d3f85b
  

  popovaan

  and

  ilya-lavrenov

  authored
  1d3f85b

• Update src/cpp/src/scheduler.hpp

  Show description for 3fa02d0
  

  popovaan

  and

  ilya-lavrenov

  authored
  3fa02d0

• Update scheduler.hpp

  ilya-lavrenov

  authored
  a0456d8

• Merge branch 'master' into dynamic_kv_cache_allocation

  ilya-lavrenov

  authored
  e393d3d