Code

CLA fixes and improvements#2

Code

Open

KeremTurgutlu wants to merge 7 commits intomainAnswerDotAI/transformers:mainfrom cla_qwenAnswerDotAI/transformers:cla_qwenCopy head branch name to clipboard

+730-62

Commits

Commits on Oct 22, 2024

• qwen cla, fp8 kv init

  

  Kerem Turgutlu

  committed
  5a61919

Commits on Oct 24, 2024

• query only rotary emb

  

  Kerem Turgutlu

  committed
  9748c74

• llama cla

  

  Kerem Turgutlu

  committed
  c4d6d55

• Merge pull request #1 from AnswerDotAI/cla_qwen

  Show description for b8c2055
  austinvhuang

  authored
  b8c2055

Commits on Oct 25, 2024

• scalar not supported by FSDP, add CLA training test

  

  Ubuntu

  committed
  ca26596

• detach shared kv by default

  

  Ubuntu

  committed
  b1af4ca

Commits on Oct 28, 2024

• gradual unfreeze

  

  Kerem Turgutlu

  committed
  c4fd5cd