1131 Commits

Author	SHA1	Message	Date
Ryan Dick	5e2351f3bf	Fix FLUX output image clamping. And a few other minor fixes to make inference work with the full bfloat16 FLUX transformer model.	2024-08-21 09:10:22 -04:00
Brandon Rising	d705c3cf0e	Select dev/schnell based on state dict, use correct max seq len based on dev/schnell, and shift in inference, separate vae flux params into separate config	2024-08-21 09:10:20 -04:00
Brandon Rising	115f350f6f	Install sub directories with folders correctly, ensure consistent dtype of tensors in flux pipeline and vae	2024-08-21 09:09:39 -04:00
Brandon Rising	be6cb2c07c	Working inference node with quantized bnb nf4 checkpoint	2024-08-21 09:09:39 -04:00
Brandon Rising	4fb5529493	Remove unused param on _run_vae_decoding in flux text to image	2024-08-21 09:09:39 -04:00
Brandon Rising	b43ee0b837	Add nf4 bnb quantized format	2024-08-21 09:09:39 -04:00
Brandon Rising	3312fe8fc4	Run ruff, setup initial text to image node	2024-08-21 09:09:39 -04:00
Brandon Rising	01a2449dae	Add backend functions and classes for Flux implementation, Update the way flux encoders/tokenizers are loaded for prompt encoding, Update way flux vae is loaded	2024-08-21 09:09:37 -04:00
Brandon Rising	cfe9d0ce0a	Some UI cleanup, regenerate schema	2024-08-21 09:08:22 -04:00
Brandon Rising	46b6314482	Run Ruff	2024-08-21 09:06:38 -04:00
Brandon Rising	46d5107ff1	Run Ruff	2024-08-21 09:06:38 -04:00
Brandon Rising	6ea1278d22	Manage quantization of models within the loader	2024-08-21 09:06:34 -04:00
Brandon Rising	f425d3aa3c	Setup flux model loading in the UI	2024-08-21 09:04:37 -04:00
Ryan Dick	d7a39a4d67	WIP on moving from diffusers to FLUX	2024-08-21 08:59:19 -04:00
Ryan Dick	0e96794c6e	LLM.int8() quantization is working, but still some rough edges to solve.	2024-08-21 08:59:19 -04:00
Ryan Dick	23a7328a66	Clean up NF4 implementation.	2024-08-21 08:59:19 -04:00
Ryan Dick	c3cf8c3b6b	NF4 inference working	2024-08-21 08:59:19 -04:00
Ryan Dick	3ba60e1656	Split a FluxTextEncoderInvocation out from the FluxTextToImageInvocation. This has the advantage that we benfit from automatic caching when the prompt isn't changed.	2024-08-21 08:59:19 -04:00
Ryan Dick	cdd47b657b	Make quantized loading fast for both T5XXL and FLUX transformer.	2024-08-21 08:59:19 -04:00
Ryan Dick	e8fb8f4d12	Make float16 inference work with FLUX on 24GB GPU.	2024-08-21 08:59:19 -04:00
Ryan Dick	9381211508	Add support for 8-bit quantizatino of the FLUX T5XXL text encoder.	2024-08-21 08:59:19 -04:00
Ryan Dick	8cce4a40d4	Make 8-bit quantization save/reload work for the FLUX transformer. Reload is still very slow with the current optimum.quanto implementation.	2024-08-21 08:59:19 -04:00
Ryan Dick	4833746698	Minor improvements to FLUX workflow.	2024-08-21 08:59:19 -04:00
Ryan Dick	8b9bf55bba	Got FLUX schnell working with 8-bit quantization. Still lots of rough edges to clean up.	2024-08-21 08:59:19 -04:00
Ryan Dick	7b199fed4f	Use the FluxPipeline.encode_prompt() api rather than trying to run the two text encoders separately.	2024-08-21 08:59:18 -04:00
Ryan Dick	13513465c8	First draft of FluxTextToImageInvocation.	2024-08-21 08:59:18 -04:00
blessedcoolant	4f8a4b0f22	Merge branch 'main' into depth_anything_v2	2024-08-03 00:38:57 +05:30
Ryan Dick	981475a624	Merge branch 'main' into ryan/grounded-sam	2024-08-01 20:30:35 +02:00
Ryan Dick	27ac61a4fb	Expose all model options in the GroundingDinoInvocation and the SegmentAnythingInvocation.	2024-08-01 14:23:32 -04:00
Ryan Dick	675ffc2757	Remove BoundingBoxInvocation field name overrides.	2024-08-01 14:05:44 -04:00
Ryan Dick	44b21f10f1	Add a pydantic model_validator to BoundingBoxField to check the validity of the coords.	2024-08-01 14:00:57 -04:00
Ryan Dick	c6d49e8b1f	Shorten SegmentAnythingInvocation and GroundingDinoInvocatino docstrings, since they are used as the invocation descriptions in the UI.	2024-08-01 10:17:42 -04:00
Ryan Dick	e6a512aa86	(minor) Tweak order of mask operations.	2024-08-01 10:12:24 -04:00
Ryan Dick	c3a6a6fb22	Rename SegmentAnythingModelInvocation -> SegmentAnythingInvocation.	2024-08-01 10:00:36 -04:00
Ryan Dick	b9dc3460ba	Rename SegmentAnythingModel -> SegmentAnythingPipeline.	2024-08-01 09:57:47 -04:00
blessedcoolant	f5cfdcf32d	feat: Add BoundingBox Primitive Node	2024-08-01 04:09:08 +05:30
Ryan Dick	b5832768dc	Return a MaskOutput from SegmentAnythingModelInvocation. And add a MaskTensorToImageInvocation.	2024-07-31 17:16:14 -04:00
Ryan Dick	5a9173f766	Merge branch 'main' into stalker-modular_lora	2024-07-31 15:13:22 -04:00
blessedcoolant	332bc9da5b	fix: Update depth anything node default to v2	2024-07-31 23:52:29 +05:30
blessedcoolant	daf899f9c4	fix: Move the manual image resizing out of the depth anything pipeline	2024-07-31 23:38:12 +05:30
blessedcoolant	13fb2d1f49	fix: Add Depth Anything V2 as a new option ... It is also now the default in the UI replacing Depth Anything V1 small	2024-07-31 23:29:43 +05:30
Ryan Dick	fca119773b	Split invokeai/backend/image_util/segment_anything/ dir into grounding_dino/ and segment_anything/	2024-07-31 12:28:47 -04:00
Ryan Dick	0193267a53	Split GroundedSamInvocation into GroundingDinoInvocation and SegmentAnythingModelInvocation.	2024-07-31 12:20:23 -04:00
Ryan Dick	9f448fecb7	Move invokeai/backend/grounded_sam -> invokeai/backend/image_util/grounded_sam	2024-07-31 10:00:30 -04:00
Ryan Dick	bcd1483a14	Re-order GroundedSAMInvocation._to_numpy_masks(...) to do slightly more work on the GPU.	2024-07-31 09:51:14 -04:00
Ryan Dick	e206890e25	Use staticmethods rather than inner functions for the Grounding DINO and SAM model loaders.	2024-07-31 09:28:52 -04:00
Ryan Dick	0a7048f650	(minor) Simplify GroundedSAMInvocation._merge_masks(...).	2024-07-31 08:58:51 -04:00
Ryan Dick	e8ecf5e155	(minor) Move apply_polygon_refinement condition up a layer.	2024-07-31 08:50:56 -04:00
Ryan Dick	cec7399366	(minor) Use a new variable name to satisfy type checks.	2024-07-31 08:27:01 -04:00
Ryan Dick	bdae81e429	(minor) Simplify GroundedSAMInvocation._filter_detections()	2024-07-31 08:25:19 -04:00