This was a difficult merge because both PR #1108 and #1243 made
changes to obscure parts of the diffusion code.
- prompt weighting, merging and cross-attention working
- cross-attention does not work with runwayML inpainting
model, but weighting and merging are tested and working
- CLI command parsing code rewritten in order to get embedded
quotes right
- --hires now works with runwayML inpainting
- --embiggen does not work with runwayML and will give an error
- Added an --invert option to invert masks applied to inpainting
- Updated documentation
- change default model back to 1.4
- remove --fnformat from canonicalized dream prompt arguments
(not needed for image reproducibility)
- add -tm to canonicalized dream prompt arguments
(definitely needed for image reproducibility)
- The plms sampler now works with custom inpainting model
- Quashed bug that was causing generation on normal models to fail (oops!)
- Can now generate non-square images with custom inpainting model
Credits for advice and assistance during porting:
@any-winter-4079 (http://github.com/any-winter-4079)
@db3000 (Danny Beer http://github.com/db3000)
This is still a work in progress but seems functional. It supports
inpainting, txt2img and img2img on the ddim and k* samplers (plms
still needs work, but I know what to do).
To test this, get the file `sd-v1-5-inpainting.ckpt' from
https://huggingface.co/runwayml/stable-diffusion-inpainting and place it
at `models/ldm/stable-diffusion-v1/sd-v1-5-inpainting.ckpt`
Launch invoke.py with --model inpainting-1.5 and proceed as usual.
Caveats:
1. The inpainting model takes about 800 Mb more memory than the standard
1.5 model. This model will not work on 4 GB cards.
2. The inpainting model is temperamental. It wants you to describe the
entire scene and not just the masked area to replace. So if you want
to replace the parrot on a man's shoulder with a crow, the prompt
"crow" may fail. Try "man with a crow on shoulder" instead. The
symptom of a failed inpainting is that the area will be erased and
replaced with background.
3. This has not been tested well. Please report bugs.
- This is a merge of the final version of PR #1218 "Inpainting
Improvements"
Various merge conflicts made it easier to commit directly.
Author: Kyle0654
Co-Author: lstein
- This is a merge of the final version of PR #1218 "Inpainting
Improvements"
Various merge conflicts made it easier to commit directly.
Author: Kyle0654
Co-Author: lstein
Now you can activate the Hugging Face `diffusers` library safety check
for NSFW and other potentially disturbing imagery.
To turn on the safety check, pass --safety_checker at the command
line. For developers, the flag is `safety_checker=True` passed to
ldm.generate.Generate(). Once the safety checker is turned on, it
cannot be turned off unless you reinitialize a new Generate object.
When the safety checker is active, suspect images will be blurred and
a warning icon is added. There is also a warning message printed in
the CLI, but it can be a little hard to see because of its positioning
in the output stream.
There is a slight but noticeable delay when the safety checker runs.
Note that invisible watermarking is *not* currently implemented. The
watermark code distributed by the CompViz distribution uses a library
that does not seem to be able to retrieve the watermarks it creates,
and it does not appear that Hugging Face `diffusers` or other SD
distributions are doing any watermarking.
1. If tensors are passed to inpaint as init_image and/or init_mask, then
the post-generation image fixup code will be skipped.
2. Post-generation image fixup will work with either a black and white "L"
or "RGB" mask, or an "RGBA" mask.
- pass a PIL.Image to img2img and inpaint rather than tensor
- To support clipseg, inpaint needs to accept an "L" or "1" format
mask. Made the appropriate change.
To add a VAE autoencoder to an existing model:
1. Download the appropriate autoencoder and put it into
models/ldm/stable-diffusion
Note that you MUST use a VAE that was written for the
original CompViz Stable Diffusion codebase. For v1.4,
that would be the file named vae-ft-mse-840000-ema-pruned.ckpt
that you can download from https://huggingface.co/stabilityai/sd-vae-ft-mse-original
2. Edit config/models.yaml to contain the following stanza, modifying `weights`
and `vae` as required to match the weights and vae model file names. There is
no requirement to rename the VAE file.
~~~
stable-diffusion-1.4:
weights: models/ldm/stable-diffusion-v1/sd-v1-4.ckpt
description: Stable Diffusion v1.4
config: configs/stable-diffusion/v1-inference.yaml
vae: models/ldm/stable-diffusion-v1/vae-ft-mse-840000-ema-pruned.ckpt
width: 512
height: 512
~~~
3. Alternatively from within the `invoke.py` CLI, you may use the command
`!editmodel stable-diffusion-1.4` to bring up a simple editor that will
allow you to add the path to the VAE.
4. If you are just installing InvokeAI for the first time, you can also
use `!import_model models/ldm/stable-diffusion/sd-v1.4.ckpt` instead
to create the configuration from scratch.
5. That's it!
* Removed duplicate fix_func for MPS
* add support for loading VAE autoencoders
To add a VAE autoencoder to an existing model:
1. Download the appropriate autoencoder and put it into
models/ldm/stable-diffusion
Note that you MUST use a VAE that was written for the
original CompViz Stable Diffusion codebase. For v1.4,
that would be the file named vae-ft-mse-840000-ema-pruned.ckpt
that you can download from https://huggingface.co/stabilityai/sd-vae-ft-mse-original
2. Edit config/models.yaml to contain the following stanza, modifying `weights`
and `vae` as required to match the weights and vae model file names. There is
no requirement to rename the VAE file.
~~~
stable-diffusion-1.4:
weights: models/ldm/stable-diffusion-v1/sd-v1-4.ckpt
description: Stable Diffusion v1.4
config: configs/stable-diffusion/v1-inference.yaml
vae: models/ldm/stable-diffusion-v1/vae-ft-mse-840000-ema-pruned.ckpt
width: 512
height: 512
~~~
3. Alternatively from within the `invoke.py` CLI, you may use the command
`!editmodel stable-diffusion-1.4` to bring up a simple editor that will
allow you to add the path to the VAE.
4. If you are just installing InvokeAI for the first time, you can also
use `!import_model models/ldm/stable-diffusion/sd-v1.4.ckpt` instead
to create the configuration from scratch.
5. That's it!
* ported code refactor changes from PR #1221
- pass a PIL.Image to img2img and inpaint rather than tensor
- To support clipseg, inpaint needs to accept an "L" or "1" format
mask. Made the appropriate change.
* minor fixes to inpaint code
1. If tensors are passed to inpaint as init_image and/or init_mask, then
the post-generation image fixup code will be skipped.
2. Post-generation image fixup will work with either a black and white "L"
or "RGB" mask, or an "RGBA" mask.
Co-authored-by: wfng92 <43742196+wfng92@users.noreply.github.com>
The k_samplers come with a "karras" noise schedule which performs
very well at low step counts but becomes noisy at higher ones.
This commit introduces a threshold (currently 30 steps) at which the
k samplers will switch over from using karras to the older model
noise schedule.
To add a VAE autoencoder to an existing model:
1. Download the appropriate autoencoder and put it into
models/ldm/stable-diffusion
Note that you MUST use a VAE that was written for the
original CompViz Stable Diffusion codebase. For v1.4,
that would be the file named vae-ft-mse-840000-ema-pruned.ckpt
that you can download from https://huggingface.co/stabilityai/sd-vae-ft-mse-original
2. Edit config/models.yaml to contain the following stanza, modifying `weights`
and `vae` as required to match the weights and vae model file names. There is
no requirement to rename the VAE file.
~~~
stable-diffusion-1.4:
weights: models/ldm/stable-diffusion-v1/sd-v1-4.ckpt
description: Stable Diffusion v1.4
config: configs/stable-diffusion/v1-inference.yaml
vae: models/ldm/stable-diffusion-v1/vae-ft-mse-840000-ema-pruned.ckpt
width: 512
height: 512
~~~
3. Alternatively from within the `invoke.py` CLI, you may use the command
`!editmodel stable-diffusion-1.4` to bring up a simple editor that will
allow you to add the path to the VAE.
4. If you are just installing InvokeAI for the first time, you can also
use `!import_model models/ldm/stable-diffusion/sd-v1.4.ckpt` instead
to create the configuration from scratch.
5. That's it!
Ironically, the black and white mask file generated by the
`invoke> !mask` command could not be passed as the mask to
`img2img`. This is now fixed and the documentation updated.
- code for committing config changes to models.yaml now in module
rather than in invoke script
- model marked "default" is now loaded if model not specified on
command line
- uncache changed models when edited, so that they reload properly
- removed liaon from models.yaml and added stable-diffusion-1.5
- The !mask command takes an image path, a text prompt, and
(optionally) a masking threshold. It creates a mask over the region
indicated by the prompt, and outputs several files that show which
regions will be masked by the chosen prompt and threshold.
- The mask images should not be passed directly to img2img because
they are designed for visualization only. Instead, use the
--text_mask option to pass the selected prompt and threshold.
- See docs/features/INPAINTING.md for details.
- The directory "models" in the main InvokeAI directory was conflicting
with loading "models.clipseg". To fix this issue, I have renamed the
models.clipseg to clipseg_models.clipseg, and applied this change to
the 'models-rename' branch of invoke-ai's fork of clipseg.
attention is parsed but ignored, blends old syntax doesn't work,
conjunctions are parsed but ignored, the only part that's used
here is the new .blend() syntax and cross-attention control
using .swap()
commit 9bb0b5d0036c4dffbb72ce11e097fae4ab63defd
Author: Damian at mba <damian@frey.NOSPAMco.nz>
Date: Sat Oct 15 23:43:41 2022 +0200
undo local_files_only stuff
commit eed93f5d30c34cfccaf7497618ae9af17a5ecfbb
Author: Damian at mba <damian@frey.NOSPAMco.nz>
Date: Sat Oct 15 23:40:37 2022 +0200
Revert "Merge branch 'development-invoke' into fix-prompts"
This reverts commit 7c40892a9f184f7e216f14d14feb0411c5a90e24, reversing
changes made to e3f2dd62b0548ca6988818ef058093a4f5b022f2.
commit f06d6024e345c69e6d5a91ab5423925a68ee95a7
Author: Damian at mba <damian@frey.NOSPAMco.nz>
Date: Thu Oct 13 23:30:16 2022 +0200
more efficiently handle multiple conditioning
commit 5efdfcbcd980ce6202ab74e7f90e7415ce7260da
Merge: b9c0dc5 ac08bb6
Author: Damian at mba <damian@frey.NOSPAMco.nz>
Date: Thu Oct 13 14:51:01 2022 +0200
Merge branch 'optional-disable-karras-schedule' into fix-prompts
commit ac08bb6fd25e19a9d35cf6c199e66500fb604af1
Author: Damian at mba <damian@frey.NOSPAMco.nz>
Date: Thu Oct 13 14:50:43 2022 +0200
append '*use_model_sigmas*' to prompt string to use model sigmas
commit 70d8c05a3ff329409f76204f4af94e55d468ab8b
Author: Damian at mba <damian@frey.NOSPAMco.nz>
Date: Thu Oct 13 12:12:17 2022 +0200
make karras scheduling switchable
commit d60df54f69 replaced the model's
own scheduling with karras scheduling. this has changed image generation
(seems worse now?)
this commit wraps the change in a bool.
commit b9c0dc5f1a658a0e6c3936000e9ae559e1c7a1db
Author: Damian at mba <damian@frey.NOSPAMco.nz>
Date: Wed Oct 12 20:16:00 2022 +0200
add test of more complex conjunction
commit 9ac0c15cc0d7b5f6df3289d3ad474260972a17be
Author: Damian at mba <damian@frey.NOSPAMco.nz>
Date: Wed Oct 12 17:18:25 2022 +0200
improve comments
commit ad33bce60590b87b2a93e90f16dc9d3e935d04a5
Author: Damian at mba <damian@frey.NOSPAMco.nz>
Date: Wed Oct 12 17:04:46 2022 +0200
put back thresholding stuff
commit 4852c698a325049834ba0d4b358f07210bc7171a
Author: Damian at mba <damian@frey.NOSPAMco.nz>
Date: Wed Oct 12 14:25:02 2022 +0200
notes on improving conjunction efficiency
commit a53bb1e5b68025d09642b935ae6a9a015cfaf2d6
Author: Damian at mba <damian@frey.NOSPAMco.nz>
Date: Wed Oct 12 14:14:33 2022 +0200
optional weights support for Conjunction
commit fec79ab15e4f0c84dd61cb1b45a5e6a72ae4aaeb
Author: Damian at mba <damian@frey.NOSPAMco.nz>
Date: Wed Oct 12 12:07:27 2022 +0200
fix blend error and log parsing output
commit 1f751c2a039f9c97af57b18e0f019512631d5a25
Author: Damian at mba <damian@frey.NOSPAMco.nz>
Date: Wed Oct 12 10:33:33 2022 +0200
fix broken euler sampler
commit 02f8148d17efe4b6bde8d29b827092a0626363ee
Author: Damian at mba <damian@frey.NOSPAMco.nz>
Date: Wed Oct 12 10:24:20 2022 +0200
cleanup prompt parser
commit 8028d49ae6c16c0d6ec9c9de9c12d56c32201421
Author: Damian at mba <damian@frey.NOSPAMco.nz>
Date: Wed Oct 12 10:14:18 2022 +0200
explicit conjunction, improve flattening logic
commit 8a1710892185f07eb77483f7edae0fc4d6bbb250
Author: Damian at mba <damian@frey.NOSPAMco.nz>
Date: Tue Oct 11 22:59:30 2022 +0200
adapt multi-conditioning to also work with ddim
commit 53802a839850d0d1ff017c6bafe457c4bed750b0
Author: Damian at mba <damian@frey.NOSPAMco.nz>
Date: Tue Oct 11 22:31:42 2022 +0200
unconditioning is also fancy-prompt-syntaxable
commit 7c40892a9f184f7e216f14d14feb0411c5a90e24
Merge: e3f2dd6 dbe0da4
Author: Damian at mba <damian@frey.NOSPAMco.nz>
Date: Tue Oct 11 21:39:54 2022 +0200
Merge branch 'development-invoke' into fix-prompts
commit e3f2dd62b0548ca6988818ef058093a4f5b022f2
Merge: eef0e48 06f542e
Author: Damian at mba <damian@frey.NOSPAMco.nz>
Date: Tue Oct 11 21:38:09 2022 +0200
Merge remote-tracking branch 'upstream/development' into fix-prompts
commit eef0e484c2eaa1bd4e0e0b1d3f8d7bba38478144
Author: Damian at mba <damian@frey.NOSPAMco.nz>
Date: Tue Oct 11 21:26:25 2022 +0200
fix run-on paren-less attention, add some comments
commit fd29afdf0e9f5e0cdc60239e22480c36ca0aaeca
Author: Damian at mba <damian@frey.NOSPAMco.nz>
Date: Tue Oct 11 21:03:02 2022 +0200
python 3.9 compatibility
commit 26f7646eef7f39bc8f7ce805e747df0f723464da
Author: Damian at mba <damian@frey.NOSPAMco.nz>
Date: Tue Oct 11 20:58:42 2022 +0200
first pass connecting PromptParser to conditioning
commit ae53dff3796d7b9a5e7ed30fa1edb0374af6cd8d
Author: Damian at mba <damian@frey.NOSPAMco.nz>
Date: Tue Oct 11 20:51:15 2022 +0200
update frontend dist
commit 9be4a59a2d76f49e635474b5984bfca826a5dab4
Author: Damian at mba <damian@frey.NOSPAMco.nz>
Date: Tue Oct 11 19:01:39 2022 +0200
fix issues with correctness checking FlattenedPrompt
commit 3be212323eab68e72a363a654124edd9809e4cf0
Author: Damian at mba <damian@frey.NOSPAMco.nz>
Date: Tue Oct 11 18:43:16 2022 +0200
parsing nested seems to work pretty ok
commit acd73eb08cf67c27cac8a22934754321256f56a9
Author: Damian at mba <damian@frey.NOSPAMco.nz>
Date: Tue Oct 11 18:26:17 2022 +0200
wip introducing FlattenedPrompt class
commit 71698d5c7c2ac855b690d8ef67e8830148c59eda
Author: Damian at mba <damian@frey.NOSPAMco.nz>
Date: Tue Oct 11 15:59:42 2022 +0200
recursive attention weighting seems to actually work
commit a4e1ec6b20deb7cc0cd12737bdbd266e56144709
Author: Damian at mba <damian@frey.NOSPAMco.nz>
Date: Tue Oct 11 15:06:24 2022 +0200
now apparently almost supported nested attention
commit da76fd1ddf22a3888cdc08fd4fed38d8b178e524
Author: Damian at mba <damian@frey.NOSPAMco.nz>
Date: Tue Oct 11 13:23:37 2022 +0200
wip prompt parsing
commit dbe0da4572c2ac22f26a7afd722349a5680a9e47
Author: Kyle Schouviller <kyle0654@hotmail.com>
Date: Mon Oct 10 22:32:35 2022 -0700
Adding node-based invocation apps
commit 8f2a2ffc083366de74d7dae471b50b6f98a7c5f8
Author: Damian at mba <damian@frey.NOSPAMco.nz>
Date: Mon Oct 10 19:03:18 2022 +0200
fix merge issues
commit 73118dee2a8f4891700756e014caf1c9ca629267
Merge: fd00844 12413b0
Author: Damian at mba <damian@frey.NOSPAMco.nz>
Date: Mon Oct 10 12:42:48 2022 +0200
Merge remote-tracking branch 'upstream/development' into fix-prompts
commit fd0084413541013c2cf71e006af0392719bef53d
Author: Damian at mba <damian@frey.NOSPAMco.nz>
Date: Mon Oct 10 12:39:38 2022 +0200
wip prompt parsing
commit 0be9363db9307859d2b65cffc6af01f57d7873a4
Author: Damian at mba <damian@frey.NOSPAMco.nz>
Date: Mon Oct 10 03:20:06 2022 +0200
better +/- attention parsing
commit 5383f691874a58ab01cda1e4fac6cf330146526a
Author: Damian at mba <damian@frey.NOSPAMco.nz>
Date: Mon Oct 10 02:27:47 2022 +0200
prompt parser seems to work
commit 591d098a33ce35462428d8c169501d8ed73615ab
Author: Damian at mba <damian@frey.NOSPAMco.nz>
Date: Sun Oct 9 20:25:37 2022 +0200
supports weighting unconditioning, cross-attention with |
commit 7a7220563aa05a2980235b5b908362f66b728309
Author: Damian at mba <damian@frey.NOSPAMco.nz>
Date: Sun Oct 9 18:15:56 2022 +0200
i think cross attention might be working?
commit 951ed391e7126bff228c18b2db304ad28d59644a
Author: Damian at mba <damian@frey.NOSPAMco.nz>
Date: Sun Oct 9 16:04:54 2022 +0200
weighted CFG denoiser working with a single item
commit ee532a0c2827368c9e45a6a5f3975666402873da
Author: Damian at mba <damian@frey.NOSPAMco.nz>
Date: Sun Oct 9 06:33:40 2022 +0200
wip probably doesn't work or compile
commit 14654bcbd207b9ca28a6cbd37dbd967d699b062d
Author: Damian at mba <damian@frey.NOSPAMco.nz>
Date: Fri Oct 7 18:11:48 2022 +0200
use tan() to calculate embedding weight for <1 attentions
commit 1a8e76b31aa5abf5150419ebf3b29d4658d07f2b
Author: Damian at mba <damian@frey.NOSPAMco.nz>
Date: Fri Oct 7 16:14:54 2022 +0200
fix bad math.max reference
commit f697ff896875876ccaa1e5527405bdaa7ed27cde
Author: Damian at mba <damian@frey.NOSPAMco.nz>
Date: Fri Oct 7 15:55:57 2022 +0200
respect http[s]x protocol when making socket.io middleware
commit 41d3dd4eeae8d4efb05dfb44fc6d8aac5dc468ab
Author: Damian at mba <damian@frey.NOSPAMco.nz>
Date: Fri Oct 7 13:29:54 2022 +0200
fractional weighting works, by blending with prompts excluding the word
commit 087fb6dfb3e8f5e84de8c911f75faa3e3fa3553c
Author: Damian at mba <damian@frey.NOSPAMco.nz>
Date: Fri Oct 7 10:52:03 2022 +0200
wip doing weights <1 by averaging with conditioning absent the lower-weighted fragment
commit 3c49e3f3ec7c18dc60f3e18ed2f7f0d97aad3a47
Author: Damian at mba <damian@frey.NOSPAMco.nz>
Date: Fri Oct 7 10:36:15 2022 +0200
notate CFGDenoiser, perhaps
commit d2bcf1bb522026ebf209ad0103f6b370383e5070
Author: Damian at mba <damian@frey.NOSPAMco.nz>
Date: Thu Oct 6 05:04:47 2022 +0200
hack blending syntax to test attention weighting more extensively
commit 94904ef2cf917f74ec23ef7a570e12ff8255b048
Author: Damian at mba <damian@frey.NOSPAMco.nz>
Date: Thu Oct 6 04:56:37 2022 +0200
conditioning works, apparently
commit 7c6663ddd70f665fd1308b6dd74f92ca393a8df5
Author: Damian at mba <damian@frey.NOSPAMco.nz>
Date: Thu Oct 6 02:20:24 2022 +0200
attention weighting, definitely works in positive direction
commit 5856d453a9b020bc1a28ff643ae1f58c12c9be73
Author: Damian at mba <damian@frey.NOSPAMco.nz>
Date: Tue Oct 4 19:02:14 2022 +0200
wip bubbling weights down
commit a2ed14fd9b7d3cb36b6c5348018b364c76d1e892
Author: Damian at mba <damian@frey.NOSPAMco.nz>
Date: Tue Oct 4 17:35:39 2022 +0200
bring in changes from PC
test prompt:
"a cat sitting on a car {a dog sitting on a car}" -W 384 -H 256 -s 10 -S 12346 -A k_euler
note that substition of dog for cat is currently hard-coded (ksampler.py
line 43-44)
On the command line, the new option is --text_mask or -tm.
Example:
```
invoke> a baseball -I /path/to/still_life.png -tm orange
```
This will find the orange fruit in the still life painting and replace
it with an image of a baseball.
- In CLI: the argument is --png_compression <0..9> (-z<0..9>)
- In API, pass `compress_level` to PngWriter.save_image_and_prompt_to_png()
Compression ranges from 0 (no compression) to 9 (maximum compression).
Default value is 6 (as specified by Pillow package).
This addresses an issue first raised in #652.
- --inpaint_replace 0.X will cause inpainting to ignore what is under
the masked region with a strength ranging from 0 (don't ignore at all)
to 1.0 (ignore completely)
- sync with upstream development
- update docs
- add a `--inpaint_replace` option that fills masked regions with
latent noise. This allows radical changes to inpainted regions
at the cost of losing context.
- fix up readline, arg processing and metadata writing to accommodate
this change
- fixed bug in storage and retrieval of variations, discovered incidentally
during testing
- update documentation
- Error checks for invalid model
- Add !del_model command to invoke.py
- Add del_model() method to model_cache
- Autocompleter kept in sync with model addition/subtraction.
At step counts greater than ~75, the ksamplers start producing noisy
images when using the Karras noise schedule. This PR reverts to using
the model's own noise schedule, which eliminates the problem at the
cost of slowing convergence at lower step counts.
This PR also introduces a new CLI `--save_intermediates <n>' argument,
which will save every nth intermediate image into a subdirectory
named `intermediates/<image_prefix>'.
Addresses issue #1083.
At step counts greater than ~75, the ksamplers start producing noisy
images when using the Karras noise schedule. This PR reverts to using
the model's own noise schedule, which eliminates the problem at the
cost of slowing convergence at lower step counts.
This PR also introduces a new CLI `--save_intermediates <n>' argument,
which will save every nth intermediate image into a subdirectory
named `intermediates/<image_prefix>'.
Addresses issue #1083.
- !import_model <path/to/model/weights> will import a new model,
prompt the user for its name and description, write it to the
models.yaml file, and load it.
- !edit_model <model_name> will bring up a previously-defined model
and prompt the user to edit its descriptive fields.
Example of !import_model
<pre>
invoke> <b>!import_model models/ldm/stable-diffusion-v1/model-epoch08-float16.ckpt</b>
>> Model import in process. Please enter the values needed to configure this model:
Name for this model: <b>waifu-diffusion</b>
Description of this model: <b>Waifu Diffusion v1.3</b>
Configuration file for this model: <b>configs/stable-diffusion/v1-inference.yaml</b>
Default image width: <b>512</b>
Default image height: <b>512</b>
>> New configuration:
waifu-diffusion:
config: configs/stable-diffusion/v1-inference.yaml
description: Waifu Diffusion v1.3
height: 512
weights: models/ldm/stable-diffusion-v1/model-epoch08-float16.ckpt
width: 512
OK to import [n]? <b>y</b>
>> Caching model stable-diffusion-1.4 in system RAM
>> Loading waifu-diffusion from models/ldm/stable-diffusion-v1/model-epoch08-float16.ckpt
| LatentDiffusion: Running in eps-prediction mode
| DiffusionWrapper has 859.52 M params.
| Making attention of type 'vanilla' with 512 in_channels
| Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
| Making attention of type 'vanilla' with 512 in_channels
| Using faster float16 precision
</pre>
Example of !edit_model
<pre>
invoke> <b>!edit_model waifu-diffusion</b>
>> Editing model waifu-diffusion from configuration file ./configs/models.yaml
description: <b>Waifu diffusion v1.4beta</b>
weights: models/ldm/stable-diffusion-v1/<b>model-epoch10-float16.ckpt</b>
config: configs/stable-diffusion/v1-inference.yaml
width: 512
height: 512
>> New configuration:
waifu-diffusion:
config: configs/stable-diffusion/v1-inference.yaml
description: Waifu diffusion v1.4beta
weights: models/ldm/stable-diffusion-v1/model-epoch10-float16.ckpt
height: 512
width: 512
OK to import [n]? y
>> Caching model stable-diffusion-1.4 in system RAM
>> Loading waifu-diffusion from models/ldm/stable-diffusion-v1/model-epoch10-float16.ckpt
...
</pre>
This commit "reverts" the new API changes by extracting the old
functionality into new files.
The work is based on the commit `803a51d5adca7e6e28491fc414fd3937bee7cb79`
PngWriter regained PromptFormatter as old server used that.
`server_legacy.py` is the old server that `dream.py` used.
Finally `legacy_api.py` is what `dream.py` used to be at the mentioned
commit.
One manually run test has been added in order to be able to test
compatibility with the old API, currently just testing that the API
endpoint works the same way + the image hash is the same as it used to
be before.
- This PR enables two new commands in the invoke.py script
!models -- list the available models and their cache status
!switch <model> -- switch to the indicated model
Example:
invoke> !models
laion400m not loaded Latent Diffusion LAION400M model
stable-diffusion-1.4 active Stable Diffusion inference model version 1.4
waifu-1.3 cached Waifu anime model version 1.3
invoke> !switch waifu-1.3
>> Caching model stable-diffusion-1.4 in system RAM
>> Retrieving model waifu-1.3 from system RAM cache
The name and descriptions of the models are taken from
`config/models.yaml`. A future enhancement to `model_cache.py` will be
to enable new model stanzas to be added to the file
programmatically. This will be useful for the WebGUI.
More details:
- Use fast switching algorithm described in PR #948
- Models are selected using their configuration stanza name
given in models.yaml.
- To avoid filling up CPU RAM with cached models, this PR
implements an LRU cache that monitors available CPU RAM.
- The caching code allows the minimum value of available RAM
to be adjusted, but invoke.py does not currently have a
command-line argument that allows you to set it. The
minimum free RAM is arbitrarily set to 2 GB.
- Add optional description field to configs/models.yaml
Unrelated fixes:
- Added ">>" to CompViz model loading messages in order to make user experience
more consistent.
- When generating an image greater than defaults, will only warn about possible
VRAM filling the first time.
- Fixed bug that was causing help message to be printed twice. This involved
moving the import line for the web backend into the section where it is
called.
Coauthored by: @ArDiouscuros
- the prompt argument comes before the optional arguments
- usage statement shows 'invoke>' rather than 'invoke.py'
- use pydoc pager to help display long help message
- the prompt argument comes before the optional arguments
- usage statement shows 'invoke>' rather than 'invoke.py'
- use pydoc pager to help display long help message
- rename dream.py to invoke.py
- create a compatibility script named dream.py that execs() invoke.py
- redo documentation
- change help message in args
- this does **not** rename the libraries, which are still ldm.dream.util, etc
This reverts commit 5f42d08945.
This fix was intended to solve issue #939, in which ESRGAN generates
dark images when upscaling 4X on certain GTX cards. However, the fix
apparently causes conflicts with some versions of the ESRGAN library,
and this fix will have to wait until after release of 2.0.
- txt2img2img back to using DDIM as img2img sampler; results produced
by some k* samplers are just not reliable enough for good user
experience
- img2img progress message clarifies why img2img steps taken != steps requested
- warn of potential problems when user tries to run img2img on a small init image
- Added support for pyreadline3 so that Window users can benefit.
- Added the !search command to search the history for a matching string:
~~~
!search puppies
[20] puppies at the food bowl -Ak_lms
[54] house overrun by hungry puppies -C20 -s100
~~~
- Added the !clear command to clear the in-memory and on-disk
command history.
- embiggen needs to use ddim sampler due to low step count
- --hires_fix option needs to be written to log and command string
- fix call signature of _init_image_mask()
- img2img confirmed working with all samplers
- inpainting working on ddim & plms. Changes to k-diffusion
module seem to be needed for inpainting support.
- switched k-diffuser noise schedule to original karras schedule,
which reduces the step number needed for good results
-if readline.set_auto_history() is not implemented, as in pyreadline3, will fall
back gracefully to automatic history saving. The only issue with this is that
-!history commands will be recorded in the history.
-!fetch on missing file no longer crashes script
-!history is now one of the autocomplete commands
-.dream_history now stored in output directory rather than ~user directory.
An important limitation of the last feature is that the history is
loaded and saved to the .dream_history file in the --outdir directory
specified at script launch time. It is not swapped around when the
--outdir is changed during the session.
Add message about interpolation size
Fix crash if sampler not set to DDIM, change parameter name to hires_fix
Hi res mode fix duplicates with img2img scaling
-if readline.set_auto_history() is not implemented, as in pyreadline3, will fall
back gracefully to automatic history saving. The only issue with this is that
-!history commands will be recorded in the history.
-!fetch on missing file no longer crashes script
-!history is now one of the autocomplete commands
-.dream_history now stored in output directory rather than ~user directory.
An important limitation of the last feature is that the history is
loaded and saved to the .dream_history file in the --outdir directory
specified at script launch time. It is not swapped around when the
--outdir is changed during the session.
- When --save_orig *not* provided during image generation with
upscaling/face fixing, an extra image file was being created. This
PR fixes the problem.
- Also generalizes the tab autocomplete for image paths such that
autocomplete searches the output directory for all path-modifying
options except for --outdir.
- normalized how filenames are written out when postprocessing invoked
- various fixes of bugs encountered during testing
- updated documentation
- updated help text
- Enhance tab completion functionality
- Each of the switches that read a filepath (e.g. --init_img) will trigger file path completion. The
-S switch will display a list of recently-used seeds.
- Added new !fetch command to retrieve the metadata from a previously-generated image and populate the
readline linebuffer with the appropriate editable command to regenerate.
- Added new !history command to display previous commands and reload them for modification.
- The !fetch and !fix commands both autocomplete *and* search automatically through the current
outdir for files.
- The completer maintains a list of recently used seeds and will try to autocomplete them.
- normalized how filenames are written out when postprocessing invoked
- various fixes of bugs encountered during testing
- updated documentation
- updated help text
- Enhance tab completion functionality
- Each of the switches that read a filepath (e.g. --init_img) will trigger file path completion. The
-S switch will display a list of recently-used seeds.
- Added new !fetch command to retrieve the metadata from a previously-generated image and populate the
readline linebuffer with the appropriate editable command to regenerate.
- Added new !history command to display previous commands and reload them for modification.
- The !fetch and !fix commands both autocomplete *and* search automatically through the current
outdir for files.
- The completer maintains a list of recently used seeds and will try to autocomplete them.
- args.py will now attempt to return a metadata-containing Args
object using the following methods:
1. By looking for the 'sd-metadata' tag in the PNG info
2. By looking from the 'Dream' tag
3. As a last resort, fetch the seed from the filename and assume
defaults for all other options.
Build the base generator in same place and way as other generators to reduce the chance of missed arguments in the future.
Fixes crash with display in-progress images, though note the feature still doesn't work for other reasons.
1. Add ldm/dream/restoration/__init__.py file that was inadvertently not
committed earlier.
2. Add '.' to sys.path to address weird mac problem reported in #723
- Adapted from PR #489, author Dominic Letz [https://github.com/dominicletz]
- Too many upstream changes to merge, so frankensteined it in.
- Added support for !fix syntax
- Added documentation
- The seed printed needs to be the one generated prior to the
initial noising operation. To do this, I added a new "first_seed"
argument to the image callback in dream.py.
- Closes#641
- modify strength of embiggen to reduce tiling ghosts
- normalize naming of postprocessed files (could improve more to avoid
name collisions)
- move restoration modules under ldm.dream
- supports gfpgan, esrgan, codeformer and embiggen
- To use:
dream> !fix ./outputs/img-samples/000056.292144555.png -ft gfpgan -U2 -G0.8
dream> !fix ./outputs/img-samples/000056.292144555.png -ft codeformer -G 0.8
dream> !fix ./outputs/img-samples/000056.29214455.png -U4
dream> !fix ./outputs/img-samples/000056.292144555.png -embiggen 1.5
The first example invokes gfpgan to fix faces and esrgan to upscale.
The second example invokes codeformer to fix faces, no upscaling
The third example uses esrgan to upscale 4X
The four example runs embiggen to enlarge 1.5X
- This is very preliminary work. There are some anomalies to note:
1. The syntax is non-obvious. I would prefer something like:
!fix esrgan,gfpgan
!fix esrgan
!fix embiggen,codeformer
However, this will require refactoring the gfpgan and embiggen
code.
2. Images generated using gfpgan, esrgan or codeformer all are named
"xxxxxx.xxxxxx.postprocessed.png" and the original is saved.
However, the prefix is a new one that is not related to the
original.
3. Images generated using embiggen are named "xxxxx.xxxxxxx.png",
and once again the prefix is new. I'm not sure whether the
prefix should be aligned with the original file's prefix or not.
Probably not, but opinions welcome.
Allowed values are 'auto', 'float32', 'autocast', 'float16'. If not specified or 'auto' a working precision is automatically selected based on the torch device.
Context: #526
Deprecated --full_precision / -F
Tested on both cuda and cpu by calling scripts/dream.py without arguments and checked the auto configuration worked. With --precision=auto/float32/autocast/float16 it performs as expected, either working or failing with a reasonable error. Also checked Img2Img.
- modify strength of embiggen to reduce tiling ghosts
- normalize naming of postprocessed files (could improve more to avoid
name collisions)
- move restoration modules under ldm.dream
- supports gfpgan, esrgan, codeformer and embiggen
- To use:
dream> !fix ./outputs/img-samples/000056.292144555.png -ft gfpgan -U2 -G0.8
dream> !fix ./outputs/img-samples/000056.292144555.png -ft codeformer -G 0.8
dream> !fix ./outputs/img-samples/000056.29214455.png -U4
dream> !fix ./outputs/img-samples/000056.292144555.png -embiggen 1.5
The first example invokes gfpgan to fix faces and esrgan to upscale.
The second example invokes codeformer to fix faces, no upscaling
The third example uses esrgan to upscale 4X
The four example runs embiggen to enlarge 1.5X
- This is very preliminary work. There are some anomalies to note:
1. The syntax is non-obvious. I would prefer something like:
!fix esrgan,gfpgan
!fix esrgan
!fix embiggen,codeformer
However, this will require refactoring the gfpgan and embiggen
code.
2. Images generated using gfpgan, esrgan or codeformer all are named
"xxxxxx.xxxxxx.postprocessed.png" and the original is saved.
However, the prefix is a new one that is not related to the
original.
3. Images generated using embiggen are named "xxxxx.xxxxxxx.png",
and once again the prefix is new. I'm not sure whether the
prefix should be aligned with the original file's prefix or not.
Probably not, but opinions welcome.
* Support color correction for img2img and inpainting, avoiding the shift to magenta seen when running images through img2img repeatedly.
* Fix docs for color correction
* add --init_color to prompt reconstruction
* For best results, the --init_color option should point to the *very first* image used in the sequence of img2img operations. Otherwise color correction will skew towards cyan.
Co-authored-by: Lincoln Stein <lincoln.stein@gmail.com>
Fixes:
File "stable-diffusion/ldm/modules/diffusionmodules/model.py", line 37, in nonlinearity
return x*torch.sigmoid(x)
RuntimeError: CUDA out of memory. Tried to allocate 1.56 GiB [..]
Now up to 1536x1280 is possible on 8GB VRAM.
Also remove unused SiLU class.
Apply ~6% speedup by moving * self.scale to earlier on a smaller tensor.
When we have enough VRAM don't make a useless zeros tensor.
Switch between cuda/mps/cpu based on q.device.type to allow cleaner per architecture future optimizations.
For cuda and cpu keep VRAM usage and faster slicing consistent.
For cpu use smaller slices. Tested ~20% faster on i7, 9.8 to 7.7 s/it.
Fix = typo to self.mem_total >= 8 in einsum_op_mps_v2 as per #582 discussion.
- fixes no closing quote in pretty-printed dream_prompt string
- removes unecessary -f switch when txt2img used
In addition, this commit does an experimental commenting-out of the
random.seed() call in the variation-generating part of ldm.dream.generator.base.
This fixes the problem of two calls that use the same seed and -v0.1
generating different images (#641). However, it does not fix the issue
of two images generated using the same seed and -VXXXXXX being
different.
* Implements rudimentary api
* Fixes blocking in API
* Adds UI to monorepo > src/frontend/
* Updates frontend/README
* Reverts conda env name to `ldm`
* Fixes environment yamls
* CORS config for testing
* Fixes LogViewer position
* API WID
* Adds actions to image viewer
* Increases vite chunkSizeWarningLimit to 1500
* Implements init image
* Implements state persistence in localStorage
* Improve progress data handling
* Final build
* Fixes mimetypes error on windows
* Adds error logging
* Fixes bugged img2img strength component
* Adds sourcemaps to dev build
* Fixes missing key
* Changes connection status indicator to text
* Adds ability to serve other hosts than localhost
* Adding Flask API server
* Removes source maps from config
* Fixes prop transfer
* Add missing packages and add CORS support
* Adding API doc
* Remove defaults from openapi doc
* Adds basic error handling for server config query
* Mostly working socket.io implementation.
* Fixes bug preventing mask upload
* Fixes bug with sampler name not written to metadata
* UI Overhaul, numerous fixes
Co-authored-by: Kyle Schouviller <kyle0654@hotmail.com>
Co-authored-by: Lincoln Stein <lincoln.stein@gmail.com>
* Feature complete for #266 with exception of several small deviations:
1. initial image and model weight hashes use full sha256 hash rather than first 8 digits
2. Initialization parameters for post-processing steps not provided
3. Uses top-level "images" tags for both a single image and a grid of images. This change was suggested in a comment.
* Added scripts/sd_metadata.py to retrieve and print metadata from PNG files
* New ldm.dream.args.Args class is a namespace like object which holds all defaults and can be modified during exection to hold current settings.
* Modified dream.py and server.py to accommodate Args class.
This change makes it so any API clients can show the same error as what
happens in the terminal where you run the API. Useful for various WebUIs
to display more helpful error messages to users.
Co-authored-by: CapableWeb <capableweb@domain.com>
* Update dream.py. k_euler_a and k_dpm_2_a M1 fix
Make results reproducible (so runs with the same seed produce the same result).
Implements fix by @wbowling referenced in https://github.com/lstein/stable-diffusion/issues/397#issuecomment-1240679294
* Update dream.py. Remove import torch from dream.py
* generate.py: k_euler_a and k_dpm_2_a M1 fix#579
Co-authored-by: Lincoln Stein <lincoln.stein@gmail.com>
* Refactor generate.py and dream.py
* config file path (models.yaml) is parsed inside Generate() to simplify
API
* Better handling of keyboard interrupts in file loading mode vs
interactive
* Removed oodles of unused variables.
* move nonfunctional inpainting out of the scripts directory
* fix ugly ddim tqdm formatting
* fix embiggen breakage, formatting fixes
* fix web server handling of rel and abs outdir paths
* Can now specify either a relative or absolute path for outdir
* Outdir path does not need to be inside the stable-diffusion directory
* Closes security hole that allowed user to read any file within
stable-diffusion (eek!)
* Closes#536
* revert inadvertent change of conda env name (#528)
* Refactor generate.py and dream.py
* config file path (models.yaml) is parsed inside Generate() to simplify
API
* Better handling of keyboard interrupts in file loading mode vs
interactive
* Removed oodles of unused variables.
* move nonfunctional inpainting out of the scripts directory
* fix ugly ddim tqdm formatting
Code cleanup and attention.py einsum_ops update for M1 16-32GB performance.
Expected: On par with fastest ever from 8 to 128GB for 512x512. Allows large images.
When running on just cpu (intel), a call to torch.layer_norm would error with RuntimeError: expected scalar type BFloat16 but found Float
Fix buggy device handling in model.py.
Tested with scripts/dream.py --full_precision on just cpu on intel laptop. Works but slow at ~10s/it.
* Add Embiggen automation
* Make embiggen_tiles masking more intelligent and count from one (at least for the user), rewrite sections of Embiggen README, fix various typos throughout README
* drop duplicate log message
commit 1c649e4663
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Mon Sep 12 13:29:16 2022 -0400
fix torchvision dependency version #511
commit 4d197f699e
Merge: a3e07fb190ba78
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Mon Sep 12 07:29:19 2022 -0400
Merge branch 'development' of github.com:lstein/stable-diffusion into development
commit a3e07fb84a
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Mon Sep 12 07:28:58 2022 -0400
fix grid crash
commit 9fa1f31bf2
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Mon Sep 12 07:07:05 2022 -0400
fix opencv and realesrgan dependencies in mac install
commit 190ba78960
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Mon Sep 12 01:50:58 2022 -0400
Update requirements-mac.txt
Fixed dangling dash on last line.
commit 25d9ccc509
Author: Any-Winter-4079 <50542132+Any-Winter-4079@users.noreply.github.com>
Date: Mon Sep 12 03:17:29 2022 +0200
Update model.py
commit 9cdf3aca7d
Author: Any-Winter-4079 <50542132+Any-Winter-4079@users.noreply.github.com>
Date: Mon Sep 12 02:52:36 2022 +0200
Update attention.py
Performance improvements to generate larger images in M1 #431
Update attention.py
Added dtype=r1.dtype to softmax
commit 49a96b90d8
Author: Mihai <299015+mh-dm@users.noreply.github.com>
Date: Sat Sep 10 16:58:07 2022 +0300
~7% speedup (1.57 to 1.69it/s) from switch to += in ldm.modules.attention. (#482)
Tested on 8GB eGPU nvidia setup so YMMV.
512x512 output, max VRAM stays same.
commit aba94b85e8
Author: Niek van der Maas <mail@niekvandermaas.nl>
Date: Fri Sep 9 15:01:37 2022 +0200
Fix macOS `pyenv` instructions, add code block highlight (#441)
Fix: `anaconda3-latest` does not work, specify the correct virtualenv, add missing init.
commit aac5102cf3
Author: Henry van Megen <h.vanmegen@gmail.com>
Date: Thu Sep 8 05:16:35 2022 +0200
Disabled debug output (#436)
Co-authored-by: Henry van Megen <hvanmegen@gmail.com>
commit 0ab5a36464
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Sun Sep 11 17:19:46 2022 -0400
fix missing lines in outputs
commit 5e433728b5
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Sun Sep 11 16:20:14 2022 -0400
upped max_steps in v1-finetune.yaml and fixed TI docs to address #493
commit 7708f4fb98
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Sun Sep 11 16:03:37 2022 -0400
slight efficiency gain by using += in attention.py
commit b86a1deb00
Author: blessedcoolant <54517381+blessedcoolant@users.noreply.github.com>
Date: Mon Sep 12 07:47:12 2022 +1200
Remove print statement styling (#504)
Co-authored-by: Lincoln Stein <lincoln.stein@gmail.com>
commit 4951e66103
Author: chromaticist <mhostick@gmail.com>
Date: Sun Sep 11 12:44:26 2022 -0700
Adding support for .bin files from huggingface concepts (#498)
* Adding support for .bin files from huggingface concepts
* Updating documentation to include huggingface .bin info
commit 79b445b0ca
Merge: a323070 f7662c1
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Sun Sep 11 15:39:38 2022 -0400
Merge branch 'development' of github.com:lstein/stable-diffusion into development
commit a323070a4d
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Sun Sep 11 15:28:57 2022 -0400
update requirements for new location of gfpgan
commit f7662c1808
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Sun Sep 11 15:00:24 2022 -0400
update requirements for changed location of gfpgan
commit 93c242c9fb
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Sun Sep 11 14:47:58 2022 -0400
make gfpgan_model_exists flag available to web interface
commit c7c6cd7735
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Sun Sep 11 14:43:07 2022 -0400
Update UPSCALE.md
New instructions needed to accommodate fact that the ESRGAN and GFPGAN packages are now installed by environment.yaml.
commit 77ca83e103
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Sun Sep 11 14:31:56 2022 -0400
Update CLI.md
Final documentation tweak.
commit 0ea145d188
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Sun Sep 11 14:29:26 2022 -0400
Update CLI.md
More doc fixes.
commit 162285ae86
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Sun Sep 11 14:28:45 2022 -0400
Update CLI.md
Minor documentation fix
commit 37c921dfe2
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Sun Sep 11 14:26:41 2022 -0400
documentation enhancements
commit 4f72cb44ad
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Sun Sep 11 13:05:38 2022 -0400
moved the notebook files into their own directory
commit 878ef2e9e0
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Sun Sep 11 12:58:06 2022 -0400
documentation tweaks
commit 4923118610
Merge: 16f6a67defafc0
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Sun Sep 11 12:51:25 2022 -0400
Merge branch 'development' of github.com:lstein/stable-diffusion into development
commit defafc0e8e
Author: Dominic Letz <dominic@diode.io>
Date: Sun Sep 11 18:51:01 2022 +0200
Enable upscaling on m1 (#474)
commit 16f6a6731d
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Sun Sep 11 12:47:26 2022 -0400
install GFPGAN inside SD repository in order to fix 'dark cast' issue #169
commit 0881d429f2
Author: blessedcoolant <54517381+blessedcoolant@users.noreply.github.com>
Date: Mon Sep 12 03:52:43 2022 +1200
Docs Update (#466)
Authored-by: @blessedcoolant
Co-authored-by: Lincoln Stein <lincoln.stein@gmail.com>
commit 9a29d442b4
Author: Gérald LONLAS <gerald@lonlas.com>
Date: Sun Sep 11 23:23:18 2022 +0800
Revert "Add 3x Upscale option on the Web UI (#442)" (#488)
This reverts commit f8a540881c.
commit d301836fbd
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Sun Sep 11 10:52:19 2022 -0400
can select prior output for init_img using -1, -2, etc
commit 70aa674e9e
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Sun Sep 11 10:34:06 2022 -0400
merge PR #495 - keep using float16 in ldm.modules.attention
commit 8748370f44
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Sun Sep 11 10:22:32 2022 -0400
negative -S indexing recovers correct previous seed; closes issue #476
commit 839e30e4b8
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Sun Sep 11 10:02:44 2022 -0400
improve CUDA VRAM monitoring
extra check that device==cuda before getting VRAM stats
commit bfb2781279
Author: tildebyte <337875+tildebyte@users.noreply.github.com>
Date: Sat Sep 10 10:15:56 2022 -0400
fix(readme): add note about updating env via conda (#475)
commit 5c43988862
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Sat Sep 10 10:02:43 2022 -0400
reduce VRAM memory usage by half during model loading
* This moves the call to half() before model.to(device) to avoid GPU
copy of full model. Improves speed and reduces memory usage dramatically
* This fix contributed by @mh-dm (Mihai)
commit 99122708ca
Merge: 817c4a2ecc6b75
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Sat Sep 10 09:54:34 2022 -0400
Merge branch 'development' of github.com:lstein/stable-diffusion into development
commit 817c4a26de
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Sat Sep 10 09:53:27 2022 -0400
remove -F option from normalized prompt; closes#483
commit ecc6b75a3e
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Sat Sep 10 09:53:27 2022 -0400
remove -F option from normalized prompt
commit 723d074442
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Fri Sep 9 18:49:51 2022 -0400
Allow ctrl c when using --from_file (#472)
* added ansi escapes to highlight key parts of CLI session
* adjust exception handling so that ^C will abort when reading prompts from a file
commit 75f633cda8
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Fri Sep 9 12:03:45 2022 -0400
re-add new logo
commit 10db192cc4
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Fri Sep 9 09:26:10 2022 -0400
changes to dogettx optimizations to run on m1
* Author @any-winter-4079
* Author @dogettx
Thanks to many individuals who contributed time and hardware to
benchmarking and debugging these changes.
commit c85ae00b33
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Thu Sep 8 23:57:45 2022 -0400
fix bug which caused seed to get "stuck" on previous image even when UI specified -1
commit 1b5aae3ef3
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Thu Sep 8 22:36:47 2022 -0400
add icon to dream web server
commit 6abf739315
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Thu Sep 8 22:25:09 2022 -0400
add favicon to web server
commit db825b8138
Merge: 33874baafee7f9
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Thu Sep 8 22:17:37 2022 -0400
Merge branch 'deNULL-development' into development
commit 33874bae8d
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Thu Sep 8 22:16:29 2022 -0400
Squashed commit of the following:
commit afee7f9cea
Merge: 6531446171f8db
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Thu Sep 8 22:14:32 2022 -0400
Merge branch 'development' of github.com:deNULL/stable-diffusion into deNULL-development
commit 171f8db742
Author: Denis Olshin <me@denull.ru>
Date: Thu Sep 8 03:15:20 2022 +0300
saving full prompt to metadata when using web ui
commit d7e67b62f0
Author: Denis Olshin <me@denull.ru>
Date: Thu Sep 8 01:51:47 2022 +0300
better logic for clicking to make variations
commit afee7f9cea
Merge: 6531446171f8db
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Thu Sep 8 22:14:32 2022 -0400
Merge branch 'development' of github.com:deNULL/stable-diffusion into deNULL-development
commit 653144694f
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Thu Sep 8 20:41:37 2022 -0400
work around unexplained crash when timesteps=1000 (#440)
* work around unexplained crash when timesteps=1000
* this fix seems to work
commit c33a84cdfd
Author: blessedcoolant <54517381+blessedcoolant@users.noreply.github.com>
Date: Fri Sep 9 12:39:51 2022 +1200
Add New Logo (#454)
* Add instructions on how to install alongside pyenv (#393)
Like probably many others, I have a lot of different virtualenvs, one for each project. Most of them are handled by `pyenv`.
After installing according to these instructions I had issues with ´pyenv`and `miniconda` fighting over the $PATH of my system.
But then I stumbled upon this nice solution on SO: https://stackoverflow.com/a/73139031 , upon which I have based my suggested changes.
It runs perfectly on my M1 setup, with the anaconda setup as a virtual environment handled by pyenv.
Feel free to incorporate these instructions as you see fit.
Thanks a million for all your hard work.
* Disabled debug output (#436)
Co-authored-by: Henry van Megen <hvanmegen@gmail.com>
* Add New Logo
Co-authored-by: Håvard Gulldahl <havard@lurtgjort.no>
Co-authored-by: Henry van Megen <h.vanmegen@gmail.com>
Co-authored-by: Henry van Megen <hvanmegen@gmail.com>
Co-authored-by: Lincoln Stein <lincoln.stein@gmail.com>
commit f8a540881c
Author: Gérald LONLAS <gerald@lonlas.com>
Date: Fri Sep 9 01:45:54 2022 +0800
Add 3x Upscale option on the Web UI (#442)
commit 244239e5f6
Author: James Reynolds <magnusviri@users.noreply.github.com>
Date: Thu Sep 8 05:36:33 2022 -0600
macOS CI workflow, dream.py exits with an error, but the workflow com… (#396)
* macOS CI workflow, dream.py exits with an error, but the workflow completes.
* Files for testing
Co-authored-by: James Reynolds <magnsuviri@me.com>
Co-authored-by: Lincoln Stein <lincoln.stein@gmail.com>
commit 711d49ed30
Author: James Reynolds <magnusviri@users.noreply.github.com>
Date: Thu Sep 8 05:35:08 2022 -0600
Cache model workflow (#394)
* Add workflow that caches the model, step 1 for CI
* Change name of workflow job
Co-authored-by: James Reynolds <magnsuviri@me.com>
Co-authored-by: Lincoln Stein <lincoln.stein@gmail.com>
commit 7996a30e3a
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Thu Sep 8 07:34:03 2022 -0400
add auto-creation of mask for inpainting (#438)
* now use a single init image for both image and mask
* turn on debugging for now to write out mask and image
* add back -M option as a fallback
commit a69ca31f34
Author: elliotsayes <elliotsayes@gmail.com>
Date: Thu Sep 8 15:30:06 2022 +1200
.gitignore WebUI temp files (#430)
* Add instructions on how to install alongside pyenv (#393)
Like probably many others, I have a lot of different virtualenvs, one for each project. Most of them are handled by `pyenv`.
After installing according to these instructions I had issues with ´pyenv`and `miniconda` fighting over the $PATH of my system.
But then I stumbled upon this nice solution on SO: https://stackoverflow.com/a/73139031 , upon which I have based my suggested changes.
It runs perfectly on my M1 setup, with the anaconda setup as a virtual environment handled by pyenv.
Feel free to incorporate these instructions as you see fit.
Thanks a million for all your hard work.
* .gitignore WebUI temp files
Co-authored-by: Håvard Gulldahl <havard@lurtgjort.no>
commit 5c6b612a72
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Wed Sep 7 22:50:55 2022 -0400
fix bug that caused same seed to be redisplayed repeatedly
commit 56f155c590
Author: Johan Roxendal <johan@roxendal.com>
Date: Thu Sep 8 04:50:06 2022 +0200
added support for parsing run log and displaying images in the frontend init state (#410)
Co-authored-by: Johan Roxendal <johan.roxendal@litteraturbanken.se>
Co-authored-by: Lincoln Stein <lincoln.stein@gmail.com>
commit 41687746be
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Wed Sep 7 20:24:35 2022 -0400
added missing initialization of latent_noise to None
commit 171f8db742
Author: Denis Olshin <me@denull.ru>
Date: Thu Sep 8 03:15:20 2022 +0300
saving full prompt to metadata when using web ui
commit d7e67b62f0
Author: Denis Olshin <me@denull.ru>
Date: Thu Sep 8 01:51:47 2022 +0300
better logic for clicking to make variations
commit d1d044aa87
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Wed Sep 7 17:56:59 2022 -0400
actual image seed now written into web log rather than -1 (#428)
commit edada042b3
Author: Arturo Mendivil <60411196+artmen1516@users.noreply.github.com>
Date: Wed Sep 7 10:42:26 2022 -0700
Improve notebook and add requirements file (#422)
commit 29ab3c2028
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Wed Sep 7 13:28:11 2022 -0400
disable neonpixel optimizations on M1 hardware (#414)
* disable neonpixel optimizations on M1 hardware
* fix typo that was causing random noise images on m1
commit 7670ecc63f
Author: cody <cnmizell@gmail.com>
Date: Wed Sep 7 12:24:41 2022 -0500
add more keyboard support on the web server (#391)
add ability to submit prompts with the "enter" key
add ability to cancel generations with the "escape" key
commit dd2aedacaf
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Wed Sep 7 13:23:53 2022 -0400
report VRAM usage stats during initial model loading (#419)
commit f6284777e6
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Tue Sep 6 17:12:39 2022 -0400
Squashed commit of the following:
commit 7d1344282d942a33dcecda4d5144fc154ec82915
Merge: caf4ea3ebeb556
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Mon Sep 5 10:07:27 2022 -0400
Merge branch 'development' of github.com:WebDev9000/stable-diffusion into WebDev9000-development
commit ebeb556af9
Author: Web Dev 9000 <rirath@gmail.com>
Date: Sun Sep 4 18:05:15 2022 -0700
Fixed unintentionally removed lines
commit ff2c4b9a1b
Author: Web Dev 9000 <rirath@gmail.com>
Date: Sun Sep 4 17:50:13 2022 -0700
Add ability to recreate variations via image click
commit c012929cda
Author: Web Dev 9000 <rirath@gmail.com>
Date: Sun Sep 4 14:35:33 2022 -0700
Add files via upload
commit 02a6018992
Author: Web Dev 9000 <rirath@gmail.com>
Date: Sun Sep 4 14:35:07 2022 -0700
Add files via upload
commit eef788981c
Author: Olivier Louvignes <olivier@mg-crea.com>
Date: Tue Sep 6 12:41:08 2022 +0200
feat(txt2img): allow from_file to work with len(lines) < batch_size (#349)
commit 720e5cd651
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Mon Sep 5 20:40:10 2022 -0400
Refactoring simplet2i (#387)
* start refactoring -not yet functional
* first phase of refactor done - not sure weighted prompts working
* Second phase of refactoring. Everything mostly working.
* The refactoring has moved all the hard-core inference work into
ldm.dream.generator.*, where there are submodules for txt2img and
img2img. inpaint will go in there as well.
* Some additional refactoring will be done soon, but relatively
minor work.
* fix -save_orig flag to actually work
* add @neonsecret attention.py memory optimization
* remove unneeded imports
* move token logging into conditioning.py
* add placeholder version of inpaint; porting in progress
* fix crash in img2img
* inpainting working; not tested on variations
* fix crashes in img2img
* ported attention.py memory optimization #117 from basujindal branch
* added @torch_no_grad() decorators to img2img, txt2img, inpaint closures
* Final commit prior to PR against development
* fixup crash when generating intermediate images in web UI
* rename ldm.simplet2i to ldm.generate
* add backward-compatibility simplet2i shell with deprecation warning
* add back in mps exception, addresses @vargol comment in #354
* replaced Conditioning class with exported functions
* fix wrong type of with_variations attribute during intialization
* changed "image_iterator()" to "get_make_image()"
* raise NotImplementedError for calling get_make_image() in parent class
* Update ldm/generate.py
better error message
Co-authored-by: Kevin Gibbons <bakkot@gmail.com>
* minor stylistic fixes and assertion checks from code review
* moved get_noise() method into img2img class
* break get_noise() into two methods, one for txt2img and the other for img2img
* inpainting works on non-square images now
* make get_noise() an abstract method in base class
* much improved inpainting
Co-authored-by: Kevin Gibbons <bakkot@gmail.com>
commit 1ad2a8e567
Author: thealanle <35761977+thealanle@users.noreply.github.com>
Date: Mon Sep 5 17:35:04 2022 -0700
Fix --outdir function for web (#373)
* Fix --outdir function for web
* Removed unnecessary hardcoded path
commit 52d8bb2836
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Mon Sep 5 10:31:59 2022 -0400
Squashed commit of the following:
commit 0cd48e932f1326e000c46f4140f98697eb9bdc79
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Mon Sep 5 10:27:43 2022 -0400
resolve conflicts with development
commit d7bc8c12e0
Author: Scott McMillin <scott@scottmcmillin.com>
Date: Sun Sep 4 18:52:09 2022 -0500
Add title attribute back to img tag
commit 5397c89184
Author: Scott McMillin <scott@scottmcmillin.com>
Date: Sun Sep 4 13:49:46 2022 -0500
Remove temp code
commit 1da080b509
Author: Scott McMillin <scott@scottmcmillin.com>
Date: Sun Sep 4 13:33:56 2022 -0500
Cleaned up HTML; small style changes; image click opens image; add seed to figcaption beneath image
commit caf4ea3d89
Author: Adam Rice <adam@askadam.io>
Date: Mon Sep 5 10:05:39 2022 -0400
Add a 'Remove Image' button to clear the file upload field (#382)
* added "remove image" button
* styled a new "remove image" button
* Update index.js
commit 95c088b303
Author: Kevin Gibbons <bakkot@gmail.com>
Date: Sun Sep 4 19:04:14 2022 -0700
Revert "Add CORS headers to dream server to ease integration with third-party web interfaces" (#371)
This reverts commit 91e826e5f4.
commit a20113d5a3
Author: Kevin Gibbons <bakkot@gmail.com>
Date: Sun Sep 4 18:59:12 2022 -0700
put no_grad decorator on make_image closures (#375)
commit 0f93dadd6a
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Sun Sep 4 21:39:15 2022 -0400
fix several dangling references to --gfpgan option, which no longer exists
commit f4004f660e
Author: tildebyte <337875+tildebyte@users.noreply.github.com>
Date: Sun Sep 4 19:43:04 2022 -0400
TOIL(requirements): Split requirements to per-platform (#355)
* toil(reqs): split requirements to per-platform
Signed-off-by: Ben Alkov <ben.alkov@gmail.com>
* toil(reqs): fix for Win and Lin...
...allow pip to resolve latest torch, numpy
Signed-off-by: Ben Alkov <ben.alkov@gmail.com>
* toil(install): update reqs in Win install notebook
Signed-off-by: Ben Alkov <ben.alkov@gmail.com>
Signed-off-by: Ben Alkov <ben.alkov@gmail.com>
commit 4406fd138d
Merge: 5116c81fd7a72e
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Sun Sep 4 08:23:53 2022 -0400
Merge branch 'SebastianAigner-main' into development
Add support for full CORS headers for dream server.
commit fd7a72e147
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Sun Sep 4 08:23:11 2022 -0400
remove debugging message
commit 3a2be621f3
Merge: 91e826e5116c81
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Sun Sep 4 08:15:51 2022 -0400
Merge branch 'development' into main
commit 5116c8178c
Author: Justin Wong <1584142+wongjustin99@users.noreply.github.com>
Date: Sun Sep 4 07:17:58 2022 -0400
fix save_original flag saving to the same filename (#360)
* Update README.md with new Anaconda install steps (#347)
pip3 version did not work for me and this is the recommended way to install Anaconda now it seems
* fix save_original flag saving to the same filename
Before this, the `--save_orig` flag was not working. The upscaled/GFPGAN would overwrite the original output image.
Co-authored-by: greentext2 <112735219+greentext2@users.noreply.github.com>
commit 91e826e5f4
Author: Sebastian Aigner <SebastianAigner@users.noreply.github.com>
Date: Sun Sep 4 10:22:54 2022 +0200
Add CORS headers to dream server to ease integration with third-party web interfaces
commit 6266d9e8d6
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Sat Sep 3 15:45:20 2022 -0400
remove stray debugging message
commit 138956e516
Author: greentext2 <112735219+greentext2@users.noreply.github.com>
Date: Sat Sep 3 13:38:57 2022 -0500
Update README.md with new Anaconda install steps (#347)
pip3 version did not work for me and this is the recommended way to install Anaconda now it seems
commit 60be735e80
Author: Cora Johnson-Roberson <cora.johnson.roberson@gmail.com>
Date: Sat Sep 3 14:28:34 2022 -0400
Switch to regular pytorch channel and restore Python 3.10 for Macs. (#301)
* Switch to regular pytorch channel and restore Python 3.10 for Macs.
Although pytorch-nightly should in theory be faster, it is currently
causing increased memory usage and slower iterations:
https://github.com/lstein/stable-diffusion/pull/283#issuecomment-1234784885
This changes the environment-mac.yaml file back to the regular pytorch
channel and moves the `transformers` dep into pip for now (since it
cannot be satisfied until tokenizers>=0.11 is built for Python 3.10).
* Specify versions for Pip packages as well.
commit d0d95d3a2a
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Sat Sep 3 14:10:31 2022 -0400
make initimg appear in web log
commit b90a215000
Merge: 1eee8116270e31
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Sat Sep 3 13:47:15 2022 -0400
Merge branch 'prixt-seamless' into development
commit 6270e313b8
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Sat Sep 3 13:46:29 2022 -0400
add credit to prixt for seamless circular tiling
commit a01b7bdc40
Merge: 1eee8119d88abe
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Sat Sep 3 13:43:04 2022 -0400
add web interface for seamless option
commit 1eee8111b9
Merge: 64eca42fb857f0
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Sat Sep 3 12:33:39 2022 -0400
Merge branch 'development' of github.com:lstein/stable-diffusion into development
commit 64eca42610
Merge: 9130ad721a1f68
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Sat Sep 3 12:33:05 2022 -0400
Merge branch 'main' into development
* brings in small documentation fixes that were
added directly to main during release tweaking.
commit fb857f05ba
Author: Lincoln Stein <lincoln.stein@gmail.com>
Date: Sat Sep 3 12:07:07 2022 -0400
fix typo in docs
commit 9d88abe2ea
Author: prixt <paraxite@naver.com>
Date: Sat Sep 3 22:42:16 2022 +0900
fixed typo
commit a61e49bc97
Author: prixt <paraxite@naver.com>
Date: Sat Sep 3 22:39:35 2022 +0900
* Removed unnecessary code
* Added description about --seamless
commit 02bee4fdb1
Author: prixt <paraxite@naver.com>
Date: Sat Sep 3 16:08:03 2022 +0900
added --seamless tag logging to normalize_prompt
commit d922b53c26
Author: prixt <paraxite@naver.com>
Date: Sat Sep 3 15:13:31 2022 +0900
added seamless tiling mode and commands
