3. test and support python 3.10 4. add a dockerfile to build docker image 5. update README.md to support docker (In Progress: upload docker image)
0.2.1.post1
1. fix precision bug imported in 0.2.1, add mmlu/mmlu_pro text, fix server 413
0.2.1
1. Update documentation/README. 307 316 2. Added Multi-GPU configuration tutorial. 254 3. Consolidated installation guide. 307 4. A new Triton MLA Kernel has been introduced for Linux 294
0.2.0
1. Support Deepseek-R1 and V3 on single (24GB VRAM)/multi gpu and 382G DRAM 2. Support use dual socket
0.1.4
Bug fix 1. Fix bug that ktransformers cannot offload whole layer in cpu. 2. Update DeepseekV2‘s multi gpu yaml examples to evenly allocate layers. 3. Update Docker file. 4. Fix bug about Qwen2-57B can not loaded 5. Fix bug with 66 , add requirements for uvicorn
0.1.3
1. support internlm2.5 for 1M Prompt under 24GB VRAM and 150GB DRAM(only local_chat) 2. decrease DeepseekV2's required VRAM from 20G to 10G. 3. fix bugs as 51 52 56