Commit Graph

7452 Commits

Author SHA1 Message Date
Ryan Dick
6ca6cf713c Tidy IPAdapter. Add types, improve field/method naming. 2023-09-08 16:00:58 -04:00
Ryan Dick
3f7d5b4e0f Remove redundant IPAdapterXL class. 2023-09-08 15:46:10 -04:00
Ryan Dick
91596d9527 Re-factor IPAdapter to patch UNet in a context manager. 2023-09-08 15:39:22 -04:00
Ryan Dick
d669f0855d Comment unused IPAdapter generate(...) methods. 2023-09-08 13:12:42 -04:00
Ryan Dick
b2d5b53b5f Pass IP-Adapter conditioning via cross_attention_kwargs instead of concatenating to the text embedding. This avoids interference with other features that manipulate the text embedding (e.g. long prompts). 2023-09-08 11:47:36 -04:00
Ryan Dick
ddc148b70b Move ConditioningData and its field classes to their own file. This will allow new conditioning types to be added more cleanly without introducing circular dependencies. 2023-09-08 11:00:11 -04:00
Ryan Dick
c2d43f007b Specify the image_embedding_len in the IPAttnProcessor rather than the text embedding length. This enables the IPAttnProcessor to handle text embeddings of varying lengths. 2023-09-07 18:20:21 -04:00
Ryan Dick
7703bf2ca1 Delete IP-Adapter copies of AttnProcessor and AttnProcessor2_0, which were unmodified from diffusers. 2023-09-07 15:00:13 -04:00
Ryan Dick
23fdf0156f Clean up IP-Adapter in diffusers_pipeline.py - WIP 2023-09-06 20:42:20 -04:00
Ryan Dick
cdbf40c9b2 Revert ControlNetInvocation changes. 2023-09-06 19:30:30 -04:00
Ryan Dick
46c9dcb113 Run yarn build. 2023-09-06 17:16:01 -04:00
Ryan Dick
6df79045fa Run typegen. 2023-09-06 17:03:37 -04:00
Ryan Dick
d776e0a0a9 Split ControlField and IpAdapterField. 2023-09-06 17:03:37 -04:00
blessedcoolant
94ec3da7b5 chore: regen scheme merge 2023-09-05 15:23:16 +12:00
blessedcoolant
f44496a579 Merge branch 'main' into feat/ip-adapter 2023-09-05 15:22:15 +12:00
blessedcoolant
42ee95ee97
fix(ui): fix non-nodes validation logic being applied to nodes invoke button (#4457)
## What type of PR is this? (check all applicable)

- [ ] Refactor
- [ ] Feature
- [x] Bug Fix
- [ ] Optimization
- [ ] Documentation Update
- [ ] Community Node Submission


## Have you discussed this change with the InvokeAI team?
- [x] Yes
- [ ] No, because:

    
## Description

fix(ui): fix non-nodes validation logic being applied to nodes invoke
button

For example, if you had an invalid controlnet setup, it would prevent
you from invoking on nodes, when node validation was disabled.

## Related Tickets & Documents

<!--
For pull requests that relate or close an issue, please include them
below. 

For example having the text: "closes #1234" would connect the current
pull
request to issue 1234.  And when we merge the pull request, Github will
automatically close the issue.
-->

- Closes
https://discord.com/channels/1020123559063990373/1028661664519831552/1148431783289966603
2023-09-05 15:03:02 +12:00
blessedcoolant
b008fd4a5f
Merge branch 'main' into fix/ui/fix-invoke-button-validation 2023-09-05 15:00:39 +12:00
blessedcoolant
6b850d506a
feat: Inpaint & Outpaint Improvements (#4408)
## What type of PR is this? (check all applicable)

- [x] Feature
- [x] Optimization

## Have you discussed this change with the InvokeAI team?
- [x] Yes


## Description

# Coherence Mode

A new parameter called Coherence Mode has been added to Coherence Pass
settings. This parameter controls what kind of Coherence Pass is done
after Inpainting and Outpainting.

- Unmasked: This performs a complete unmasked image to image pass on the
entire generation.
- Mask: This performs a masked image to image pass using your input mask
as the coherence mask.
- Mask Edge [DEFAULT] - This performs as masked image to image pass on
the edges of your mask to try and clear out the seams.

# Why The Coherence Masked Modes?

One of the issues with unmasked coherence pass arises when the diffusion
process is trying to align detailed or organic objects. Because Image to
Image tends change the image a little bit even at lower strengths, this
ends up in the paste back process being slightly misaligned. By
providing the mask to the Coherence Pass, we can try to eliminate this
in those cases. While it will be impossible to address this for every
image out there, having these options will allow the user to automate a
lot of this. For everything else there's manual paint over with inpaint.

# Graph Improvements

The graphs have now been refined quite a bit. We no longer do manual
blurring of the masks anymore for outpainting. This is no longer needed
because we now dilate the mask depending on the blur size while pasting
back. As a result we got rid of quite a few nodes that were handling
this in the older graph.

The graphs are also a lot cleaner now because we now tackle Scaled
Dimensions & Coherence Mode completely independently.

Inpainting result seem very promising especially with the Mask Edge
mode.

---

# New Infill Methods [Experimental]

We are currently trying out various new infill methods to see which ones
might perform the best in outpainting. We may keep all of them or keep
none. This will be decided as we test more.

## LaMa Infill

- Renabled LaMA infill in the UI.
- We are trying to get this to work without a memory overhead.

In order to use LaMa, you need to manually download and place the LaMa
JIT model in `models/core/misc/lama/lama.pt`. You can download the JIT
model from Sanster
[here](https://github.com/Sanster/models/releases/download/add_big_lama/big-lama.pt)
and rename it to `lama.pt` or you can use the script in the original
LaMA repo to convert the base model to a JIT model yourself.

## CV2 Infill

- Added a new infilling method using CV2's Inpaint.

## Patchmatch Rescaling

Patchmatch infill input image is now downscaled and infilled. Patchmatch
can be really slow at large resolutions and this is a pretty decent way
to get around that. Additionally, downscaling might also provide a
better patch match by avoiding larger areas to be infilled with
repeating patches. But that's just the theory. Still testing it out.

## [optional] Are there any post deployment tasks we need to perform?

- If we decide to keep LaMA infill, then we will need to host the model
and update the installer to download it as a core model.
2023-09-05 14:55:30 +12:00
blessedcoolant
99fe95ab03 fix: Add validation for image_encoder model too 2023-09-05 14:49:41 +12:00
blessedcoolant
3f3e0ab9f5
Merge branch 'main' into lama-infill 2023-09-05 14:47:53 +12:00
psychedelicious
8b305651f9 fix(ui): fix non-nodes validation logic being applied to nodes invoke button 2023-09-05 12:44:39 +10:00
psychedelicious
95ecb1a0c1 fix(ip_adapter): add None to types 2023-09-05 12:30:00 +10:00
psychedelicious
bd15874cf6 feat(nodes): add control_type validation & fix types 2023-09-05 12:24:54 +10:00
Millun Atluri
52bd2bbb13
Update communityNodes.md with a few more nodes (#4444)
Adds my (@dwringer's) released nodes to the community nodes page.

## What type of PR is this? (check all applicable)

- [ ] Refactor
- [ ] Feature
- [ ] Bug Fix
- [ ] Optimization
- [X] Documentation Update
- [ ] Community Node Submission


## Have you discussed this change with the InvokeAI team?
- [X] Yes
- [ ] No, because:

      
## Have you updated all relevant documentation?
- [X] Yes
- [ ] No


## Description
Adds my released nodes -
Depth Map from Wavefront OBJ
Enhance Image
Generative Grammar-Based Prompt Nodes
Ideal Size Stepper
Image Compositor
Final Size & Orientation / Random Switch (Integers)
Text Mask (Simple 2D)
2023-09-05 12:20:33 +10:00
blessedcoolant
a9fafad5b5 chore: sync, lint & update 2023-09-05 14:17:23 +12:00
blessedcoolant
c5b9c8fc3a Merge branch 'main' into lama-infill 2023-09-05 14:16:27 +12:00
blessedcoolant
fb5ac78191 Merge branch 'lama-infill' of https://github.com/blessedcoolant/InvokeAI into lama-infill 2023-09-05 14:11:05 +12:00
blessedcoolant
871b9286d1 fix: Review changes 2023-09-05 14:10:41 +12:00
Lincoln Stein
c49b436f06 Merge branch 'lama-infill' of github.com:blessedcoolant/InvokeAI into lama-infill 2023-09-04 21:54:52 -04:00
Lincoln Stein
d2e327add9 install models/core/misc/lama/lama.pt 2023-09-04 21:54:40 -04:00
psychedelicious
2ab75bc52e feat(ui): move fp32 check to its own variable
remove a ton of extraneous checks that are easy to miss during maintenance
2023-09-05 11:51:46 +10:00
blessedcoolant
30ab81b6bb fix: Update paths so they are serializable in the nodes 2023-09-05 13:50:21 +12:00
Darren Ringer
384ad2df6a
Merge branch 'main' into patch-2 2023-09-04 21:48:17 -04:00
blessedcoolant
78195491bc fix: Make the adapter models use new local paths 2023-09-05 13:39:54 +12:00
psychedelicious
94115b5217 fix(nodes): downscale and resample_mode are not optional 2023-09-05 11:23:13 +10:00
dunkeroni
10eec546ad
Consolidate and generalize saturation/luminosity adjusters (#4425)
* Consolidated saturation/luminosity adjust.
Now allows increasing and inverting.
Accepts any color PIL format and channel designation.

* Updated docs/nodes/defaultNodes.md

* shortened tags list to channel types only

* fix typo in mode list

* split features into offset and multiply nodes

* Updated documentation

* Change invert to discrete boolean.
Previous math was unclear and had issues with 0 values.

* chore: black

* chore(ui): typegen

---------

Co-authored-by: psychedelicious <4822129+psychedelicious@users.noreply.github.com>
2023-09-05 11:18:37 +10:00
blessedcoolant
c63390f6e1 fix: Temporarily update the ControlField zod model
While we decide how to go ahead with this .
2023-09-05 12:29:05 +12:00
Darren Ringer
ac3bf81ca4
Update communityNodes.md for consistency and conciseness
Trims down a couple of my node descriptions and adjusts the formatting a little bit for consistency.
2023-09-04 20:21:48 -04:00
blessedcoolant
cbd451c610 chore: Regen Schema 2023-09-05 12:13:08 +12:00
blessedcoolant
b0f91f2e75 fix: Remove types on adapter nodes. Superseded by the decorator 2023-09-05 12:12:19 +12:00
blessedcoolant
3ac68cde66 chore: flake8 cleanup 2023-09-05 12:07:12 +12:00
blessedcoolant
a69b1cd598 chore: Add Versioning data to new adapters + update model paths 2023-09-05 11:54:50 +12:00
blessedcoolant
65a76a086b cleanup: Some basic cleanup 2023-09-05 11:54:28 +12:00
blessedcoolant
07381e5a26 cleanup: merge conflicts 2023-09-05 11:37:12 +12:00
blessedcoolant
6bb378a101 Merge branch 'main' into feat/ip-adapter 2023-09-05 11:35:19 +12:00
Darren Ringer
edd64bd537
Replace links to .py files with repo links, and consolidate some nodes
Revised links to my node py files, replacing them with links to independent repos. Additionally I consolidated some nodes together (Image and Mask Composition Pack, Size Stepper nodes).
2023-09-04 19:25:12 -04:00
Darren Ringer
8795ea8b06
Merge branch 'main' into patch-2 2023-09-04 19:19:03 -04:00
blessedcoolant
b1ef3370fa chore: Regen Schema 2023-09-05 09:56:34 +12:00
blessedcoolant
db4af7c287 Merge branch 'main' into lama-infill 2023-09-05 09:54:44 +12:00
blessedcoolant
78cc5a7825
feat(nodes): versioning (#4449)
## What type of PR is this? (check all applicable)

- [ ] Refactor
- [x] Feature
- [ ] Bug Fix
- [ ] Optimization
- [ ] Documentation Update
- [ ] Community Node Submission


## Have you discussed this change with the InvokeAI team?
- [x] Yes
- [ ] No, because:

      
## Have you updated all relevant documentation?
- [x] Yes
- [ ] No


## Description

This PR is based on #4423 and should not be merged until it is merged.

[feat(nodes): add version to node
schemas](c179d4ccb7)

The `@invocation` decorator is extended with an optional `version` arg.
On execution of the decorator, the version string is parsed using the
`semver` package (this was an indirect dependency and has been added to
`pyproject.toml`).

All built-in nodes are set with `version="1.0.0"`.

The version is added to the OpenAPI Schema for consumption by the
client.

[feat(ui): handle node
versions](03de3e4f78)

- Node versions are now added to node templates
- Node data (including in workflows) include the version of the node
- On loading a workflow, we check to see if the node and template
versions match exactly. If not, a warning is logged to console.
- The node info icon (top-right corner of node, which you may click to
open the notes editor) now shows the version and mentions any issues.
- Some workflow validation logic has been shifted around and is now
executed in a redux listener.

## Related Tickets & Documents

<!--
For pull requests that relate or close an issue, please include them
below. 

For example having the text: "closes #1234" would connect the current
pull
request to issue 1234.  And when we merge the pull request, Github will
automatically close the issue.
-->

- Closes #4393

## QA Instructions, Screenshots, Recordings

<!-- 
Please provide steps on how to test changes, any hardware or 
software specifications as well as any other pertinent information. 
-->

Loading old workflows should prompt a warning, and the node status icon
should indicate some action is needed.

## [optional] Are there any post deployment tasks we need to perform?

I've updated the default workflows:
- Bump workflow versions from 1.0 to 1.0.1
- Add versions for all nodes in the workflows
- Test workflows

[Default
Workflows.zip](https://github.com/invoke-ai/InvokeAI/files/12511911/Default.Workflows.zip)

I'm not sure where these are being stored right now @Millu
2023-09-05 09:53:46 +12:00