Skip to content

Issues: turboderp/exllamav2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

87 Open 401 Closed
87 Open 401 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

[REQUEST] Offloading a customizable number of experts into RAM for DeepSeek V3 685B?
#706 opened Dec 26, 2024 by TyraVex
3 tasks done
1
[REQUEST] Support for the new Command-r7b
#703 opened Dec 22, 2024 by ciprianveg
3 tasks done
2
[REQUEST] Sage Attention? Anyone tried it with exllama?
#702 opened Dec 21, 2024 by Ph0rk0z
3 tasks done
2
[BUG] Qwen2.5-72B-2.xxbpw/Llama-70B-2.4bpw (maybe related to KV caching code) garbage output on some specific prompts. bug Something isn't working
#697 opened Dec 14, 2024 by Originalimoc
3 tasks done
15
[BUG] lmformatenforcer integration seems to be broken on new versions bug Something isn't working
#696 opened Dec 11, 2024 by hvico
3 tasks done
2
[REQUEST] EXAONE 3.5 Support
#695 opened Dec 9, 2024 by necrogay
3 tasks done
2
[BUG] ExLlamaV2DynamicGenerator class is not multiple threads supported bug Something isn't working
#690 opened Nov 29, 2024 by UTSAV-44
3 tasks done
4
[BUG] generator.iterate() returns corrupted result objects in some cases bug Something isn't working
#689 opened Nov 29, 2024 by p-e-w
3 tasks done
8
[REQUEST] High throughput with large batch size
#686 opened Nov 26, 2024 by fzyzcjy
3 tasks done
5
[BUG] Speculative decoding regresses performance on 7900 xtx under ROCM bug Something isn't working
#685 opened Nov 25, 2024 by Mushoz
3 tasks done
1
qwen coder32b run on colab t4 bug Something isn't working
#682 opened Nov 23, 2024 by werruww
3 tasks done
10
[REQUEST] Can we have 1.0/1.5 bpw internally?
#675 opened Nov 17, 2024 by Originalimoc
3 tasks done
1
[BUG] [Qwen] Draft model produce garbage output bug Something isn't working
#674 opened Nov 14, 2024 by Nepherpitou
3 tasks done
5
[REQUEST] Convert.py: Option to skip measurement when setting 8.0/8.0
#673 opened Nov 13, 2024 by Originalimoc
3 tasks done
1
[REQUEST] Support for a Qwen based vision model
#672 opened Nov 12, 2024 by TyraVex
3 tasks done
6
[QUESTION] Does exllamav2 support no-dequant inference?
#670 opened Nov 7, 2024 by AaronZLT
1
[REQUEST] Synthetic Data generation features
#669 opened Nov 3, 2024 by AstrisCantCode
3 tasks done
3
[PAPER] New quant method with SOTA quality and speed: QTIP
#668 opened Nov 1, 2024 by TyraVex
3 tasks done
[REQUEST] Alternative way to the Pytorch environment variables on Windows to set Pytorch memory management parameters
#664 opened Oct 29, 2024 by Nexesenex
3 tasks done
5
[BUG] AMD - Out of memory errors despite having plenty of VRAM bug Something isn't working
#662 opened Oct 27, 2024 by RSAStudioGames
3 tasks done
[REQUEST] Faster 6/8-bit EXL2 quantization
#660 opened Oct 19, 2024 by grimulkan
3 tasks done
[REQUEST] Llama 3.2 Vision Support (or already exists?)
#658 opened Oct 18, 2024 by grimulkan
3 tasks done
13
[BUG] Appending-Runtime-LoRA-weights bug Something isn't working
#656 opened Oct 16, 2024 by royallavanya140
3 tasks done
3
[BUG] Convert script fails to run on master branch as of v0.2.3 bug Something isn't working
#655 opened Oct 15, 2024 by iamwavecut
3 tasks done
5
[BUG] RAM UTILISATION IS INCREASING RAPIDLY bug Something isn't working
#639 opened Sep 25, 2024 by UTSAV-44
1
ProTip! Updated in the last three days: updated:>2024-12-23.