Mirrored_Repos/InvokeAI
1
0
Fork 0
mirror of https://github.com/invoke-ai/InvokeAI synced 2024-08-30 20:32:17 +00:00
Code Issues Packages Projects Releases Wiki Activity

Update docs for T5 quantization script.

Browse Source
This commit is contained in:
Ryan Dick 2024-08-23 18:07:14 +00:00
parent 6405214940
commit 8af3c72de7
1 changed files with 6 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

11
invokeai/backend/quantization/quantize_t5_xxl_bnb_llm_int8.py
View File

@ -20,11 +20,12 @@ def load_state_dict_into_t5(model: T5EncoderModel, state_dict: dict):
def main():
# Load the FLUX transformer model onto the meta device.
model_path = Path(
# "/data/invokeai/models/.download_cache/black-forest-labs_flux.1-schnell/FLUX.1-schnell/text_encoder_2"
"/data/misc/text_encoder_2"
)
"""A script for quantizing a T5 text encoder model using the bitsandbytes LLM.int8() quantization method.
This script is primarily intended for reference. The script params (e.g. the model_path, modules_to_not_convert,
etc.) are hardcoded and would need to be modified for other use cases.
"""
model_path = Path("/data/misc/text_encoder_2")
with log_time("Intialize T5 on meta device"):
model_config = AutoConfig.from_pretrained(model_path)