Dynamic Quantiser - a way to make your own high quality dynamic quants

Dynamic Quantiser TLDR - Makes dynamic and/or custom quants of ggufs. Doesn't need data, just gguf file + llama.cpp. Minimises cosine deviation - highly correlated with minimising KLD. Fast, much better than standard quants, not as good as Unsloth on pure text, maybe as good on mixed inputs/co

中文全文翻译正在补齐，可先查看下方英文原文。

Dynamic Quantiser TLDR - Makes dynamic and/or custom quants of ggufs. Doesn't need data, just gguf file + llama.cpp. Minimises cosine deviation - highly correlated with minimising KLD. Fast, much better than standard quants, not as good as Unsloth on pure text, maybe as good on mixed inputs/co