Brandon Rising
723f3ab0a9
Add nf4 bnb quantized format
2024-08-26 20:17:50 -04:00
Brandon Rising
1bd90e0fd4
Run ruff, setup initial text to image node
2024-08-26 20:17:50 -04:00
Brandon Rising
436f18ff55
Add backend functions and classes for Flux implementation, Update the way flux encoders/tokenizers are loaded for prompt encoding, Update way flux vae is loaded
2024-08-26 20:17:50 -04:00
Brandon Rising
cde9696214
Some UI cleanup, regenerate schema
2024-08-26 20:17:50 -04:00
Brandon Rising
2d9042fb93
Run Ruff
2024-08-26 20:17:50 -04:00
Brandon Rising
9ed53af520
Run Ruff
2024-08-26 20:17:50 -04:00
Brandon Rising
56fda669fd
Manage quantization of models within the loader
2024-08-26 20:17:50 -04:00
Brandon Rising
5f59a828f9
Setup flux model loading in the UI
2024-08-26 20:17:50 -04:00
Ryan Dick
1fa6bddc89
WIP on moving from diffusers to FLUX
2024-08-26 20:17:50 -04:00
Ryan Dick
f01f56a98e
LLM.int8() quantization is working, but still some rough edges to solve.
2024-08-26 20:17:50 -04:00
Ryan Dick
99b0f79784
Clean up NF4 implementation.
2024-08-26 20:17:50 -04:00
Ryan Dick
e1eb104345
NF4 inference working
2024-08-26 20:17:50 -04:00
Ryan Dick
a52c899c6d
Split a FluxTextEncoderInvocation out from the FluxTextToImageInvocation. This has the advantage that we benfit from automatic caching when the prompt isn't changed.
2024-08-26 20:17:50 -04:00
Ryan Dick
eeabb7ebe5
Make quantized loading fast for both T5XXL and FLUX transformer.
2024-08-26 20:17:50 -04:00
Ryan Dick
3cf0365a35
Make float16 inference work with FLUX on 24GB GPU.
2024-08-26 20:17:50 -04:00
Ryan Dick
5870742bb9
Add support for 8-bit quantizatino of the FLUX T5XXL text encoder.
2024-08-26 20:17:50 -04:00
Ryan Dick
01d8c62c57
Make 8-bit quantization save/reload work for the FLUX transformer. Reload is still very slow with the current optimum.quanto implementation.
2024-08-26 20:17:50 -04:00
Ryan Dick
55a242b2d6
Minor improvements to FLUX workflow.
2024-08-26 20:17:50 -04:00
Ryan Dick
45263b339f
Got FLUX schnell working with 8-bit quantization. Still lots of rough edges to clean up.
2024-08-26 20:17:50 -04:00
Ryan Dick
3319491861
Use the FluxPipeline.encode_prompt() api rather than trying to run the two text encoders separately.
2024-08-26 20:17:50 -04:00
Ryan Dick
b39031ea53
First draft of FluxTextToImageInvocation.
2024-08-26 20:17:50 -04:00
Mary Hipp
3e7923d072
fix(api): allow updating of type for style preset
2024-08-19 16:12:39 -04:00
psychedelicious
5a24b89e54
fix(app): include style preset defaults in build
2024-08-16 21:47:06 +10:00
psychedelicious
7a3eaa8da9
feat(api): save file as prompt_templates.csv
2024-08-16 09:51:46 +10:00
Mary Hipp
599db7296f
export only user style presets
2024-08-15 16:07:32 -04:00
Mary Hipp
24f298283f
clean up, add context menu to import/download templates
2024-08-15 12:39:55 -04:00
Mary Hipp
68dac6349d
Merge remote-tracking branch 'origin/main' into maryhipp/export-presets
2024-08-15 11:21:56 -04:00
psychedelicious
60d754d1df
feat(api): tidy style presets import logic
...
- Extract parsing into utility function
- Log import errors
- Forbid extra properties on the imported data
2024-08-15 09:47:49 -04:00
psychedelicious
bcbf8b6bd8
feat(ui): revert to using {prompt} for prompt template placeholder
2024-08-15 09:47:49 -04:00
psychedelicious
356661459b
feat(api): support JSON for preset imports
...
This allows us to support Fooocus format presets.
2024-08-15 09:47:49 -04:00
psychedelicious
deb917825e
feat(api): use pydantic validation during style preset import
...
- Enforce name is present and not an empty string
- Provide empty string as default for positive and negative prompt
- Add `positive_prompt` as validation alias for `prompt` field
- Strip whitespace automatically
- Create `TypeAdapter` to validate the whole list in one go
2024-08-15 09:47:49 -04:00
Mary Hipp
2d58754789
feat(api): add endpoint to take a CSV, parse it, validate it, and create many style preset entries
2024-08-15 09:47:49 -04:00
Mary Hipp
a9014673a0
wip export
2024-08-15 09:00:11 -04:00
psychedelicious
982c266073
tidy: remove extra characters in prompt templates
2024-08-14 12:31:57 +10:00
Mary Hipp
e0d3927265
feat: add flag for allowPrivateStylePresets that shows a type field when creating a style preset
2024-08-13 14:08:54 -04:00
Mary Hipp
e5f7c2a9b7
add type safety / validation to form data payloads and allow type to be passed through api
2024-08-13 13:00:31 -04:00
Mary Hipp
b0760710d5
add the rest of default style presets, update image service to return default images correctly by name, add tooltip popover to images in UI
2024-08-13 11:33:15 -04:00
Mary Hipp
764accc921
update config docstring
2024-08-12 15:17:40 -04:00
Mary Hipp
9c732ac3b1
Merge remote-tracking branch 'origin/main' into maryhipp/style-presets
2024-08-12 14:53:45 -04:00
Mary Hipp
4837e578b2
api: update dir path for style preset images, update payload for create/update formdata
2024-08-12 12:00:14 -04:00
psychedelicious
f6b8970bd1
fix(app): create reference to events task to prevent accidental GC
...
This wasn't a problem, but it's advised in the official docs so I've done it.
2024-08-12 07:49:58 +10:00
psychedelicious
29325a7214
fix(app): use asyncio queue and existing event loop for events
...
Around the time we (I) implemented pydantic events, I noticed a short pause between progress images every 4 or 5 steps when generating with SDXL. It didn't happen with SD1.5, but I did notice that with SD1.5, we'd get 4 or 5 progress events simultaneously. I'd expect one event every ~25ms, matching my it/s with SD1.5. Mysterious!
Digging in, I found an issue is related to our use of a synchronous queue for events. When the event queue is empty, we must call `asyncio.sleep` before checking again. We were sleeping for 100ms.
Said another way, every time we clear the event queue, we have to wait 100ms before another event can be dispatched, even if it is put on the queue immediately after we start waiting. In practice, this means our events get buffered into batches, dispatched once every 100ms.
This explains why I was getting batches of 4 or 5 SD1.5 progress events at once, but not the intermittent SDXL delay.
But this 100ms wait has another effect when the events are put on the queue in intervals that don't perfectly line up with the 100ms wait. This is most noticeable when the time between events is >100ms, and can add up to 100ms delay before the event is dispatched.
For example, say the queue is empty and we start a 100ms wait. Then, immediately after - like 0.01ms later - we push an event on to the queue. We still need to wait another 99.9ms before that event will be dispatched. That's the SDXL delay.
The easy fix is to reduce the sleep to something like 0.01 seconds, but this feels kinda dirty. Can't we just wait on the queue and dispatch every event immediately? Not with the normal synchronous queue - but we can with `asyncio.Queue`.
I switched the events queue to use `asyncio.Queue` (as seen in this commit), which lets us asynchronous wait on the queue in a loop.
Unfortunately, I ran into another issue - events now felt like their timing was inconsistent, but in a different way than with the 100ms sleep. The time between pushing events on the queue and dispatching them was not consistently ~0ms as I'd expect - it was highly variable from ~0ms up to ~100ms.
This is resolved by passing the asyncio loop directly into the events service and using its methods to create the task and interact with the queue. I don't fully understand why this resolved the issue, because either way we are interacting with the same event loop (as shown by `asyncio.get_running_loop()`). I suppose there's some scheduling magic happening.
2024-08-12 07:49:58 +10:00
psychedelicious
8ecf72838d
fix(api): image downloads with correct filename
...
Closes #6730
2024-08-10 09:53:56 -04:00
psychedelicious
f66584713c
fix(api): sort OpenAPI schema properties for InvocationOutputMap
...
This makes the schema output deterministic!
2024-08-10 07:45:23 -04:00
psychedelicious
33624fc2fa
fix(api): duplicate operation id for get_image_full
...
There's a FastAPI bug that results in the OpenAPI spec outputting the same operation id for each operation when specifying multiple HTTP methods.
- Discussion: https://github.com/tiangolo/fastapi/discussions/8449
- Pending PR to fix: https://github.com/tiangolo/fastapi/pull/10694
In our case, we have a `get_image_full` endpoint that handles GET and HEAD.
This results in an invalid OpenAPI schema. A workaround is to use two route decorators for the operation handler. This works as expected - HEAD requests get the header, and GET requests get the resource. And the OpenAPI schema is valid.
2024-08-10 07:45:23 -04:00
Mary Hipp
97553a7de2
API/DB updates per PR feedback
2024-08-09 16:27:37 -04:00
Mary Hipp
5f83bb0069
update config docstring
2024-08-08 15:20:43 -04:00
Mary Hipp
0cd7cdb52e
remove send2trash
2024-08-08 15:13:36 -04:00
Mary Hipp
1d8b7e2bcf
ruff
2024-08-08 15:08:45 -04:00
Mary Hipp
3f9a674d4b
seed default presets and handle them in UI
2024-08-08 15:02:41 -04:00
Mary Hipp
4952eada87
ruff format
2024-08-08 14:22:40 -04:00
Mary Hipp
581029ebaa
ruff
2024-08-08 14:21:37 -04:00
Mary Hipp
a7b83c8b5b
Merge remote-tracking branch 'origin/main' into maryhipp/style-presets
2024-08-08 13:56:59 -04:00
Mary Hipp
9a4d075074
fix path for style_preset_images, fix png type when converting blobs to files, built view mode components
2024-08-08 12:31:20 -04:00
Kent Keirsey
408a1d6dbb
Merge branch 'main' into depth_anything_v2
2024-08-07 10:45:56 -04:00
Mary Hipp
0b0abfbe8f
clean up image implementation
2024-08-07 10:36:38 -04:00
Mary Hipp
cc96dcf0ed
style preset images
2024-08-07 09:58:27 -04:00
Mary Hipp
2604fd9fde
a whole bunch of stuff
2024-08-06 15:31:13 -04:00
psychedelicious
98db8f395b
feat(app): clean up DiskImageStorage types
2024-08-04 09:43:20 +10:00
psychedelicious
6c4cf58a31
feat(app): delete model_images instead of using send2trash
2024-08-04 09:43:20 +10:00
psychedelicious
08993c0d29
feat(app): delete images instead of using send2trash
...
Closes #6709
2024-08-04 09:43:20 +10:00
blessedcoolant
4f8a4b0f22
Merge branch 'main' into depth_anything_v2
2024-08-03 00:38:57 +05:30
Mary Hipp
217fe40d99
feat(api): add style_presets router, make sure all CRUD is working, add is_default
2024-08-02 12:29:54 -04:00
Mary Hipp
b76bf50b93
feat(db,api): create new table for style presets, build out record storage service for style presets
2024-08-01 22:20:11 -04:00
Ryan Dick
981475a624
Merge branch 'main' into ryan/grounded-sam
2024-08-01 20:30:35 +02:00
Ryan Dick
27ac61a4fb
Expose all model options in the GroundingDinoInvocation and the SegmentAnythingInvocation.
2024-08-01 14:23:32 -04:00
Ryan Dick
675ffc2757
Remove BoundingBoxInvocation field name overrides.
2024-08-01 14:05:44 -04:00
Ryan Dick
44b21f10f1
Add a pydantic model_validator to BoundingBoxField to check the validity of the coords.
2024-08-01 14:00:57 -04:00
Ryan Dick
c6d49e8b1f
Shorten SegmentAnythingInvocation and GroundingDinoInvocatino docstrings, since they are used as the invocation descriptions in the UI.
2024-08-01 10:17:42 -04:00
Ryan Dick
e6a512aa86
(minor) Tweak order of mask operations.
2024-08-01 10:12:24 -04:00
Ryan Dick
c3a6a6fb22
Rename SegmentAnythingModelInvocation -> SegmentAnythingInvocation.
2024-08-01 10:00:36 -04:00
Ryan Dick
b9dc3460ba
Rename SegmentAnythingModel -> SegmentAnythingPipeline.
2024-08-01 09:57:47 -04:00
blessedcoolant
f5cfdcf32d
feat: Add BoundingBox Primitive Node
2024-08-01 04:09:08 +05:30
Ryan Dick
b5832768dc
Return a MaskOutput from SegmentAnythingModelInvocation. And add a MaskTensorToImageInvocation.
2024-07-31 17:16:14 -04:00
Ryan Dick
5a9173f766
Merge branch 'main' into stalker-modular_lora
2024-07-31 15:13:22 -04:00
blessedcoolant
332bc9da5b
fix: Update depth anything node default to v2
2024-07-31 23:52:29 +05:30
blessedcoolant
daf899f9c4
fix: Move the manual image resizing out of the depth anything pipeline
2024-07-31 23:38:12 +05:30
blessedcoolant
13fb2d1f49
fix: Add Depth Anything V2 as a new option
...
It is also now the default in the UI replacing Depth Anything V1 small
2024-07-31 23:29:43 +05:30
Ryan Dick
fca119773b
Split invokeai/backend/image_util/segment_anything/ dir into grounding_dino/ and segment_anything/
2024-07-31 12:28:47 -04:00
Ryan Dick
0193267a53
Split GroundedSamInvocation into GroundingDinoInvocation and SegmentAnythingModelInvocation.
2024-07-31 12:20:23 -04:00
Ryan Dick
9f448fecb7
Move invokeai/backend/grounded_sam -> invokeai/backend/image_util/grounded_sam
2024-07-31 10:00:30 -04:00
Ryan Dick
bcd1483a14
Re-order GroundedSAMInvocation._to_numpy_masks(...) to do slightly more work on the GPU.
2024-07-31 09:51:14 -04:00
Ryan Dick
e206890e25
Use staticmethods rather than inner functions for the Grounding DINO and SAM model loaders.
2024-07-31 09:28:52 -04:00
Ryan Dick
0a7048f650
(minor) Simplify GroundedSAMInvocation._merge_masks(...).
2024-07-31 08:58:51 -04:00
Ryan Dick
e8ecf5e155
(minor) Move apply_polygon_refinement condition up a layer.
2024-07-31 08:50:56 -04:00
Ryan Dick
cec7399366
(minor) Use a new variable name to satisfy type checks.
2024-07-31 08:27:01 -04:00
Ryan Dick
bdae81e429
(minor) Simplify GroundedSAMInvocation._filter_detections()
2024-07-31 08:25:19 -04:00
Ryan Dick
67c32f3d6c
Fix typo: zip(..., strict=True)
2024-07-31 08:15:28 -04:00
blessedcoolant
fa3c0c81b3
Merge branch 'main' into stalker7779/fix_gradient_mask
2024-07-31 06:30:44 +05:30
blessedcoolant
18f89ed5ed
fix: Make DepthAnything work with Invoke's Model Management
2024-07-31 03:57:54 +05:30
Ryan Dick
6b10b59abe
Make GroundedSAMInvocation work with any input image mode (RGB, RGBA, grayscale).
2024-07-30 15:55:57 -04:00
Ryan Dick
918f77bce0
Move some logic from GroundedSAMInvocation to the backend classes.
2024-07-30 15:34:33 -04:00
blessedcoolant
f170697ebe
Merge branch 'main' into depth_anything_v2
2024-07-31 00:53:32 +05:30
blessedcoolant
556c6a1d84
fix: Update DepthAnything to use the transformers implementation
2024-07-31 00:51:55 +05:30
Ryan Dick
aca2a2fa13
Add mask_filter and detection_threshold options to the GroundedSAMInvocation.
2024-07-30 14:22:40 -04:00
Ryan Dick
ff6398f7d8
Add a GroundedSamInvocation for image segmentation from a text prompt (Grounding DINO + Segment Anything Model).
2024-07-30 11:12:26 -04:00
Ryan Dick
693a3eaff5
Merge branch 'main' into stalker-modular_inpaint-2
2024-07-29 10:14:45 -04:00
Ryan Dick
310719eb4c
Merge branch 'main' into stalker-modular_t2i_adapter
2024-07-28 15:30:00 -04:00
Ryan Dick
c57a7afb87
Merge branch 'main' into stalker7779/modular_seamless
2024-07-28 13:49:43 -04:00
Sergey Borisov
faa88f72bf
Make lora as separate extensions
2024-07-27 02:39:53 +03:00
