Commit Graph

5 Commits

Author SHA1 Message Date
Ryan Dick
0e96794c6e LLM.int8() quantization is working, but still some rough edges to solve. 2024-08-21 08:59:19 -04:00
Ryan Dick
23a7328a66 Clean up NF4 implementation. 2024-08-21 08:59:19 -04:00
Ryan Dick
c3cf8c3b6b NF4 inference working 2024-08-21 08:59:19 -04:00
Ryan Dick
110d58d107 NF4 loading working... I think. 2024-08-21 08:59:19 -04:00
Ryan Dick
3480e06688 wip 2024-08-21 08:59:19 -04:00