Code

Token rotation#987

Code

Open

vshampor wants to merge 15 commits intoopenvinotoolkit:masteropenvinotoolkit/openvino.genai:masterfrom vshampor:token_rotationvshampor/openvino.genai:token_rotationCopy head branch name to clipboard

+1,096-200

Commits

Commits on Dec 20, 2024

• Implement token rotation

  vshampor

  committed
  48c6c78

• Add better test IDs

  vshampor

  committed
  b684361

• Remove debug print

  vshampor

  committed
  6c1f89e

• Fill rotation coefficients before current forward step

  vshampor

  committed
  a4d9637

• Do not download whisper datasets during collection

  vshampor

  committed
  2a8faa2

• Refactor coefficient filling, do not evict/rotate at prefill

  vshampor

  committed
  da58d36

• Fix long_prompts.txt

  vshampor

  committed
  a5ef624

• Refactor to deltas + LUT

  vshampor

  committed
  ee1cc18

• Make LUT smaller

  vshampor

  committed
  a4b7bc3

• Debug prints

  vshampor

  committed
  5d60460

• Fix bugs

  vshampor

  committed
  9e393b1

• Add shape to rotation deltas

  vshampor

  committed
  9bf9aa8

• Remove debug prints, use per-block rotation

  vshampor

  committed
  92b1310

• Add tests

  vshampor

  committed
  ea64dc5

• Fix max seq len

  vshampor

  committed
  0a753e2