Mirrored_Repos/InvokeAI
1
0
Fork 0
mirror of https://github.com/invoke-ai/InvokeAI synced 2024-08-30 20:32:17 +00:00
Code Issues Packages Projects Releases Wiki Activity

Fix bug in InvokeInt8Params that was causing it to use double the necessary VRAM.

Browse Source
This commit is contained in:
Ryan Dick 2024-08-21 19:01:11 +00:00 committed by Brandon
parent fd68a2475b
commit 19a68afb3a
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
invokeai/backend/quantization/bnb_llm_int8.py
View File

@ -22,7 +22,7 @@ class InvokeInt8Params(bnb.nn.Int8Params):
return super().cuda(device) return super().cuda(device)
elif self.CB is not None and self.SCB is not None: elif self.CB is not None and self.SCB is not None:
self.data = self.data.cuda() self.data = self.data.cuda()
self.CB = self.CB.cuda() self.CB = self.data
self.SCB = self.SCB.cuda() self.SCB = self.SCB.cuda()
else: else:
# we store the 8-bit rows-major weight # we store the 8-bit rows-major weight