Commit Graph

7055 Commits

Author SHA1 Message Date
Brandon Rising
4b6d9a73ed Merge branch 'main' into feat/batch-graphs 2023-08-18 15:40:34 -04:00
Brandon Rising
3e26214b83 Add a few more endpoints for managing batches 2023-08-18 15:38:16 -04:00
Brandon Rising
0282f46c71 Add runs field for running the same batch multiple times 2023-08-18 13:41:07 -04:00
Brandon Rising
99e03fe92e Run unmodified graph if no batch data is provided 2023-08-18 13:33:09 -04:00
Martin Kristiansen
c96ae4c331 Reverting late imports to fix tests 2023-08-18 15:52:04 +10:00
Martin Kristiansen
ce465acf04 Fixed OnnxRuntimeModel import 2023-08-18 15:52:04 +10:00
Martin Kristiansen
33ee418d8c Fixing class level import 2023-08-18 15:52:04 +10:00
Martin Kristiansen
4f1008f31f Installing Flake8-pyproject in GHA workflow 2023-08-18 15:52:04 +10:00
Martin Kristiansen
6cc629e19d Adding flake8 to GHA and pre-commit. Fixing missing flake8 2023-08-18 15:52:04 +10:00
Martin Kristiansen
537ae2f901 Resolving merge conflicts for flake8 2023-08-18 15:52:04 +10:00
psychedelicious
f6db9da06c chore(ui): rename file to not cause madge to fail 2023-08-18 13:20:29 +10:00
psychedelicious
a17dbd7df6 feat(ui): improve error toast messages 2023-08-18 13:20:29 +10:00
Lincoln Stein
498d2ecc2b
allow symbolic links to be followed during autoimport (#4268)
## What type of PR is this? (check all applicable)

- [X] Feature
- [X] Bug Fix

## Have you discussed this change with the InvokeAI team?
- [X] Yes

## Have you updated all relevant documentation?
- [X] Yes

## Description

Follow symbolic links when auto importing from a directory. Previously
links to files worked, but links to directories weren’t entered during
the scanning/import process.
2023-08-17 20:31:00 -04:00
Lincoln Stein
4ebe839d54
Merge branch 'main' into bugfix/enable-links-in-autoimport 2023-08-17 18:55:45 -04:00
Lincoln Stein
bc16b50302 add followlinks to all os.walk() calls 2023-08-17 18:54:18 -04:00
Lincoln Stein
832335998f
Update 'monkeypatched' controlnet class (#4269)
## What type of PR is this? (check all applicable)

- [ ] Refactor
- [ ] Feature
- [x] Bug Fix
- [ ] Optimization
- [ ] Documentation Update
- [ ] Community Node Submission


## Have you discussed this change with the InvokeAI team?
- [ ] Yes
- [ ] No, because:

      
## Have you updated all relevant documentation?
- [ ] Yes
- [ ] No


## Description


## Related Tickets & Documents

<!--
For pull requests that relate or close an issue, please include them
below. 

For example having the text: "closes #1234" would connect the current
pull
request to issue 1234.  And when we merge the pull request, Github will
automatically close the issue.
-->

- Related Issue #
- Closes #

## QA Instructions, Screenshots, Recordings

<!-- 
Please provide steps on how to test changes, any hardware or 
software specifications as well as any other pertinent information. 
-->

## Added/updated tests?

- [ ] Yes
- [ ] No : _please replace this line with details on why tests
      have not been included_

## [optional] Are there any post deployment tasks we need to perform?
Should be removed when added in diffusers
https://github.com/huggingface/diffusers/pull/4599
2023-08-17 15:49:54 -04:00
Lincoln Stein
1102c12084
Merge branch 'main' into fix/sdxl_controlnet 2023-08-17 15:40:51 -04:00
Lincoln Stein
b5cee7d20c blackify chore 2023-08-17 15:40:15 -04:00
Brandon Rising
cb65526880 More session not found handling 2023-08-17 14:23:12 -04:00
Lincoln Stein
842eb4bb0a
Merge branch 'main' into bugfix/enable-links-in-autoimport 2023-08-17 07:20:26 -04:00
blessedcoolant
89b82b3dc4
(feat): Add Seam Painting to Canvas (1.x, 2.x & SDXL w/ Refiner) (#4292)
## What type of PR is this? (check all applicable)

- [x] Feature

## Have you discussed this change with the InvokeAI team?
- [x] Yes
      
## Description

PR to add Seam Painting back to the Canvas.

## TODO Later

While the graph works as intended, it has become extremely large and
complex. I don't know if there's a simpler way to do this. Maybe there
is but there's soo many connections and visualizing the graph in my head
is extremely difficult. We might need to create some kind of tooling for
this. Coz it's going going to get crazier.

But well works for now.
2023-08-17 21:24:39 +12:00
blessedcoolant
8923201fdf
Merge branch 'main' into seam-painting 2023-08-17 21:21:44 +12:00
mickr777
226409107b Fix for Image Deletion issue 2023-08-17 17:18:11 +10:00
blessedcoolant
ae986bf873
Report RAM usage and RAM cache statistics after each generation (#4287)
## What type of PR is this? (check all applicable)

- [X] Feature

## Have you discussed this change with the InvokeAI team?
- [X] Yes

     
## Have you updated all relevant documentation?
- [X] Yes


## Description

This PR enhances the logging of performance statistics to include RAM
and model cache information. After each generation, the following will
be logged. The new information follows TOTAL GRAPH EXECUTION TIME.

```
[2023-08-15 21:55:39,010]::[InvokeAI]::INFO --> Graph stats: 2408dbec-50d0-44a3-bbc4-427037e3f7d4
[2023-08-15 21:55:39,010]::[InvokeAI]::INFO --> Node                 Calls    Seconds VRAM Used
[2023-08-15 21:55:39,010]::[InvokeAI]::INFO --> main_model_loader        1     0.004s     0.000G
[2023-08-15 21:55:39,010]::[InvokeAI]::INFO --> clip_skip                1     0.002s     0.000G
[2023-08-15 21:55:39,010]::[InvokeAI]::INFO --> compel                   2     2.706s     0.246G
[2023-08-15 21:55:39,010]::[InvokeAI]::INFO --> rand_int                 1     0.002s     0.244G
[2023-08-15 21:55:39,011]::[InvokeAI]::INFO --> range_of_size            1     0.002s     0.244G
[2023-08-15 21:55:39,011]::[InvokeAI]::INFO --> iterate                  1     0.002s     0.244G
[2023-08-15 21:55:39,011]::[InvokeAI]::INFO --> metadata_accumulator     1     0.002s     0.244G
[2023-08-15 21:55:39,011]::[InvokeAI]::INFO --> noise                    1     0.003s     0.244G
[2023-08-15 21:55:39,011]::[InvokeAI]::INFO --> denoise_latents          1     2.429s     2.022G
[2023-08-15 21:55:39,011]::[InvokeAI]::INFO --> l2i                      1     1.020s     1.858G
[2023-08-15 21:55:39,011]::[InvokeAI]::INFO --> TOTAL GRAPH EXECUTION TIME:    6.171s
[2023-08-15 21:55:39,011]::[InvokeAI]::INFO --> RAM used by InvokeAI process: 4.50G (delta=0.10G)
[2023-08-15 21:55:39,011]::[InvokeAI]::INFO --> RAM used to load models: 1.99G
[2023-08-15 21:55:39,011]::[InvokeAI]::INFO --> VRAM in use: 0.303G
[2023-08-15 21:55:39,011]::[InvokeAI]::INFO --> RAM cache statistics:
[2023-08-15 21:55:39,011]::[InvokeAI]::INFO -->    Model cache hits: 2
[2023-08-15 21:55:39,011]::[InvokeAI]::INFO -->    Model cache misses: 5
[2023-08-15 21:55:39,011]::[InvokeAI]::INFO -->    Models cached: 5
[2023-08-15 21:55:39,011]::[InvokeAI]::INFO -->    Models cleared from cache: 0
[2023-08-15 21:55:39,011]::[InvokeAI]::INFO -->    Cache high water mark: 1.99/7.50G    
```

There may be a memory leak in InvokeAI. I'm seeing the process memory
usage increasing by about 100 MB with each generation as shown in the
example above.
2023-08-17 16:10:18 +12:00
psychedelicious
59bc9ed399 fix(backend): handle BatchSessionNotFoundException in BatchManager._process()
The internal `BatchProcessStorage.get_session()` method throws when it finds nothing, but we were not catching any exceptions.

This caused a exception when the batch manager handles a `graph_execution_state_complete` event that did not originate from a batch.

Fixed by handling the exception.
2023-08-17 13:58:11 +10:00
psychedelicious
e62d5478fd fix(backend): fix sqlite cannot commit - no transaction is active
The `commit()` was called even if we hadn't executed anything
2023-08-17 13:55:38 +10:00
psychedelicious
2cf0d61b3e
Merge branch 'main' into feat/batch-graphs 2023-08-17 13:33:17 +10:00
psychedelicious
cc3c2756bd feat(backend): rename batch changes variable
`updateSession` -> `changes`
2023-08-17 13:32:32 +10:00
psychedelicious
67cf594bb3 feat(backend): add missing types to batch_manager_storage.py 2023-08-17 13:29:19 +10:00
psychedelicious
c5b963f1a6 fix(backend): typo
`relavent` -> `relevant`
2023-08-17 12:47:58 +10:00
psychedelicious
4d2dd6bb10 feat(backend): rename BatchManager.process to _process
Just to make it clear that this is not a method on the ABC.
2023-08-17 12:47:05 +10:00
psychedelicious
7e4beab4ff feat(backend): surface BatchSessionNodeFoundException
Catch this exception in the router and return an appropriate `HTTPException`.
2023-08-17 12:45:32 +10:00
psychedelicious
e16b5f7cdc feat(backend): deserialize batch session directly
If the values from the `session_dict` are invalid, the model instantiation will fail, or if we end up with an invalid `batch_id`, the app will not run. So I think just parsing the dict directly is equivalent.

Also the LSP analyser is pleased now - no red squigglies.
2023-08-17 12:37:03 +10:00
psychedelicious
1f355d5810 feat(backend): update batch_manager_storage.py docstrings 2023-08-17 12:31:51 +10:00
psychedelicious
df7370f9d9 chore(backend): remove unused code 2023-08-17 12:16:34 +10:00
psychedelicious
5bec64d65b fix(backend): fix typings in batch_manager.py
- `batch_indicies` is `tuple[int]` not `list[int]`
- explicit `None` return values
2023-08-17 12:07:20 +10:00
psychedelicious
8cf9bd47b2 chore(backend): remove unnecessary batch validation function
The `Batch` model is fully validated by pydantic on instantiation; we do not need any validation logic for it.
2023-08-17 11:59:47 +10:00
psychedelicious
c91621b46c fix(backend): BatchProcess.batch_id is required
Providing a `default_factory` is enough for pydantic to know to create the attribute on instantiation if it's not already provided. We can then make make the typing just `str`.
2023-08-17 11:58:29 +10:00
psychedelicious
f246b236dd fix(api): fix start_batch route responses 2023-08-17 11:51:14 +10:00
Lincoln Stein
daf75a1361 blackify 2023-08-16 21:47:29 -04:00
Lincoln Stein
fe4b2d53ed Merge branch 'feat/collect-more-stats' of github.com:invoke-ai/InvokeAI into feat/collect-more-stats 2023-08-16 21:39:29 -04:00
Lincoln Stein
c39f8b478b fix misplaced ram_used and ram_changed attributes 2023-08-16 21:39:18 -04:00
Lincoln Stein
1f82d8013e
Merge branch 'main' into feat/collect-more-stats 2023-08-16 18:51:17 -04:00
Lincoln Stein
e373bfca54 fix several broken links in the installation index 2023-08-16 17:54:39 -04:00
Lincoln Stein
2ca8611723 add +/- sign in front of RAM delta 2023-08-16 15:53:01 -04:00
Brandon Rising
f7277a8b21 Run python black 2023-08-16 15:44:52 -04:00
Brandon Rising
796ee1246b Add a batch validation test 2023-08-16 15:42:45 -04:00
Brandon Rising
29fceb960d Fix batch_manager test 2023-08-16 15:33:15 -04:00
Brandon Rising
796ff34c8a Testing out Spencer's batch data structure 2023-08-16 15:21:11 -04:00
Brandon Rising
d6a5c2dbe3 Fix tests 2023-08-16 14:35:49 -04:00