Commit Graph

1608 Commits

Author SHA1 Message Date
psychedelicious
4602efd598
feat: add profiler util (#5601)
* feat(config): add profiling config settings

- `profile_graphs` enables graph profiling with cProfile
- `profiles_dir` sets the output for profiles

* feat(nodes): add Profiler util

Simple wrapper around cProfile.

* feat(nodes): use Profiler in invocation processor

* scripts: add generate_profile_graphs.sh script

Helper to generate graphs for profiles.

* pkg: add snakeviz and gprof2dot to dev deps

These are useful for profiling.

* tests: add tests for profiler util

* fix(profiler): handle previous profile not stopped cleanly

* feat(profiler): add profile_prefix config setting

The prefix is used when writing profile output files. Useful to organise profiles into sessions.

* tidy(profiler): add `_` to private API

* feat(profiler): simplify API

* feat(profiler): use child logger for profiler logs

* chore(profiler): update docstrings

* feat(profiler): stop() returns output path

* chore(profiler): fix docstring

* tests(profiler): update tests

* chore: ruff
2024-01-31 10:51:57 +00:00
Kent Keirsey
5a67bc68a1
Merge branch 'main' into depth-anything 2024-01-23 22:31:19 -06:00
maryhipp
b7d19b8130 add project as category to back-end 2024-01-24 10:59:04 +11:00
blessedcoolant
7cb49e65bd feat: Add Resolution to DepthAnything 2024-01-23 14:13:50 -06:00
blessedcoolant
f36a691219 feat: Make the depth anything small model the default 2024-01-23 14:13:50 -06:00
blessedcoolant
8f5e2cbcc7 feat: Add Depth Anything PreProcessor 2024-01-23 14:13:50 -06:00
JPPhoto
6a2856e46f Updated field descriptions 2024-01-23 02:26:30 +11:00
Jonathan
892fe62264 Add Ideal Size node to core nodes
The Ideal Size node is useful for High-Res Optimization as it gives the optimum size for creating an initial generation with minimal artifacts (duplication and other strangeness) from today's models.

After inclusion, front end graph generation can be simplified by offloading calculations for HRO initial generation to this node.
2024-01-23 02:26:30 +11:00
psychedelicious
fafe8ccc59 fix(api): typo in no_cache_staticfiles.py 2024-01-22 16:10:25 +11:00
psychedelicious
4b88cfac19 fix(api): type in no_cache_staticfiles.py 2024-01-22 16:10:25 +11:00
psychedelicious
5fa13fba36 chore: ruff 2024-01-22 16:10:25 +11:00
psychedelicious
f28f761436 fix(api): add NoCacheStaticFiles to prevent *all* caching
The previous method wasn't totally foolproof, and locales/assets were cached.

To solve this once and for all (famous last words, I know), we can subclass `StaticFiles` and use maximally strict no-caching headers to disable caching on all static files.
2024-01-22 16:10:25 +11:00
Lincoln Stein
4536e4a8b6
Model Manager Refactor: Install remote models and store their tags and other metadata (#5361)
* add basic functionality for model metadata fetching from hf and civitai

* add storage

* start unit tests

* add unit tests and documentation

* add missing dependency for pytests

* remove redundant fetch; add modified/published dates; updated docs

* add code to select diffusers files based on the variant type

* implement Civitai installs

* make huggingface parallel downloading work

* add unit tests for model installation manager

- Fixed race condition on selection of download destination path
- Add fixtures common to several model_manager_2 unit tests
- Added dummy model files for testing diffusers and safetensors downloading/probing
- Refactored code for selecting proper variant from list of huggingface repo files
- Regrouped ordering of methods in model_install_default.py

* improve Civitai model downloading

- Provide a better error message when Civitai requires an access token (doesn't give a 403 forbidden, but redirects
  to the HTML of an authorization page -- arrgh)
- Handle case of Civitai providing a primary download link plus additional links for VAEs, config files, etc

* add routes for retrieving metadata and tags

* code tidying and documentation

* fix ruff errors

* add file needed to maintain test root diretory in repo for unit tests

* fix self->cls in classmethod

* add pydantic plugin for mypy

* use TestSession instead of requests.Session to prevent any internet activity

improve logging

fix error message formatting

fix logging again

fix forward vs reverse slash issue in Windows install tests

* Several fixes of problems detected during PR review:

- Implement cancel_model_install_job and get_model_install_job routes
  to allow for better control of model download and install.
- Fix thread deadlock that occurred after cancelling an install.
- Remove unneeded pytest_plugins section from tests/conftest.py
- Remove unused _in_terminal_state() from model_install_default.
- Remove outdated documentation from several spots.
- Add workaround for Civitai API results which don't return correct
  URL for the default model.

* fix docs and tests to match get_job_by_source() rather than get_job()

* Update invokeai/backend/model_manager/metadata/fetch/huggingface.py

Co-authored-by: Ryan Dick <ryanjdick3@gmail.com>

* Call CivitaiMetadata.model_validate_json() directly

Co-authored-by: Ryan Dick <ryanjdick3@gmail.com>

* Second round of revisions suggested by @ryanjdick:

- Fix type mismatch in `list_all_metadata()` route.
- Do not have a default value for the model install job id
- Remove static class variable declarations from non Pydantic classes
- Change `id` field to `model_id` for the sqlite3 `model_tags` table.
- Changed AFTER DELETE triggers to ON DELETE CASCADE for the metadata and tags tables.
- Made the `id` field of the `model_metadata` table into a primary key to achieve uniqueness.

* Code cleanup suggested in PR review:

- Narrowed the declaration of the `parts` attribute of the download progress event
- Removed auto-conversion of str to Url in Url-containing sources
- Fixed handling of `InvalidModelConfigException`
- Made unknown sources raise `NotImplementedError` rather than `Exception`
- Improved status reporting on cached HuggingFace access tokens

* Multiple fixes:

- `job.total_size` returns a valid size for locally installed models
- new route `list_models` returns a paged summary of model, name,
  description, tags and other essential info
- fix a few type errors

* consolidated all invokeai root pytest fixtures into a single location

* Update invokeai/backend/model_manager/metadata/metadata_store.py

Co-authored-by: psychedelicious <4822129+psychedelicious@users.noreply.github.com>

* Small tweaks in response to review comments:

- Remove flake8 configuration from pyproject.toml
- Use `id` rather than `modelId` for huggingface `ModelInfo` object
- Use `last_modified` rather than `LastModified` for huggingface `ModelInfo` object
- Add `sha256` field to file metadata downloaded from huggingface
- Add `Invoker` argument to the model installer `start()` and `stop()` routines
  (but made it optional in order to facilitate use of the service outside the API)
- Removed redundant `PRAGMA foreign_keys` from metadata store initialization code.

* Additional tweaks and minor bug fixes

- Fix calculation of aggregate diffusers model size to only count the
  size of files, not files + directories (which gives different unit test
  results on different filesystems).
- Refactor _get_metadata() and _get_download_urls() to have distinct code paths
  for Civitai, HuggingFace and URL sources.
- Forward the `inplace` flag from the source to the job and added unit test for this.
- Attach cached model metadata to the job rather than to the model install service.

* fix unit test that was breaking on windows due to CR/LF changing size of test json files

* fix ruff formatting

* a few last minor fixes before merging:

- Turn job `error` and `error_type` into properties derived from the exception.
- Add TODO comment about the reason for handling temporary directory destruction
  manually rather than using tempfile.tmpdir().

* add unit tests for reporting HTTP download errors

---------

Co-authored-by: Lincoln Stein <lstein@gmail.com>
Co-authored-by: Ryan Dick <ryanjdick3@gmail.com>
Co-authored-by: psychedelicious <4822129+psychedelicious@users.noreply.github.com>
2024-01-14 19:54:53 +00:00
Ryan Dick
296c861e7d Handle bad id in log_stats(...). 2024-01-13 15:19:57 -05:00
Ryan Dick
aa45d21fd2 Reduce the number of graph_execution_manager.get(...) calls from the InvocationStatsService. 2024-01-13 15:19:57 -05:00
Ryan Dick
ac42513da9 Remove unused reset_all_stats(...). 2024-01-13 15:19:57 -05:00
Ryan Dick
e2387546fe Rename GIG -> GB. And move it to where it's being used. 2024-01-13 15:19:57 -05:00
Ryan Dick
c8929b35f0 Refactor the invocation stats service for better readability and to support reporting the execution wall time. 2024-01-13 15:19:57 -05:00
Millun Atluri
b7b376103c Update default workflows 2024-01-12 14:59:44 -05:00
Millun Atluri
08d379bb29 Update default workflows 2024-01-12 14:58:21 -05:00
Millun Atluri
74e644c4ba
Allow bfloat16 to be configurable in invoke.yaml (#5469)
* feat: allow bfloat16 to be configurable in invoke.yaml

* fix: `torch_dtype()` util

- Use `choose_precision` to get the precision string
- Do not reference deprecated `config.full_precision` flat (why does this still exist?), if a user had this enabled it would override their actual precision setting and potentially cause a lot of confusion.

---------

Co-authored-by: psychedelicious <4822129+psychedelicious@users.noreply.github.com>
2024-01-12 18:40:37 +00:00
psychedelicious
98a44d7fa1 feat(ui): update assets
- Add various brand images, organise images
- Create favicon for docs pages (light blue version of key logo)
- Rename app title to `Invoke - Community Edition`
2024-01-12 08:02:59 +11:00
psychedelicious
26b7aadd32 fix(db): fix workflows pagination math 2024-01-11 09:42:12 -05:00
Lincoln Stein
f2e8b66be4 Fix "Cannot import name 'PagingArgumentParser' error when starting textual inversion
- Closes #5395
2024-01-11 13:57:06 +11:00
psychedelicious
989aaedc7f feat(nodes): add title for cfg rescale mult on denoise_latents 2024-01-03 13:18:50 +11:00
psychedelicious
2700d0e769 fix(nodes): fix constraints/validation for controlnet
- Fix `weight` and `begin_step_percent`, the constraints were mixed up
- Add model validatort to ensure `begin_step_percent < end_step_percent`
- Bump version
2024-01-02 07:28:53 -05:00
psychedelicious
a47d91f0e7 feat(api): add max_prompts constraints 2023-12-29 08:26:14 -05:00
Millun Atluri
a6935ae7fb
Add Tiled Upscaling to default workflows 2023-12-29 12:26:50 +11:00
Millun Atluri
4e04ea0c0d fix model names to match defaults in workflows 2023-12-27 15:18:12 +11:00
Millun Atluri
60629cba3c
Merge branch 'main' into feat/db/graceful-migrate-workflows 2023-12-26 15:27:18 +11:00
Lincoln Stein
fbede84405
[feature] Download Queue (#5225)
* add base definition of download manager

* basic functionality working

* add unit tests for download queue

* add documentation and FastAPI route

* fix docs

* add missing test dependency; fix import ordering

* fix file path length checking on windows

* fix ruff check error

* move release() into the __del__ method

* disable testing of stderr messages due to issues with pytest capsys fixture

* fix unsorted imports

* harmonized implementation of start() and stop() calls in download and & install modules

* Update invokeai/app/services/download/download_base.py

Co-authored-by: Ryan Dick <ryanjdick3@gmail.com>

* replace test datadir fixture with tmp_path

* replace DownloadJobBase->DownloadJob in download manager documentation

* make source and dest arguments to download_queue.download() an AnyHttpURL and Path respectively

* fix pydantic typecheck errors in the download unit test

* ruff formatting

* add "job cancelled" as an event rather than an exception

* fix ruff errors

* Update invokeai/app/services/download/download_default.py

Co-authored-by: psychedelicious <4822129+psychedelicious@users.noreply.github.com>

* use threading.Event to stop service worker threads; handle unfinished job edge cases

* remove dangling STOP job definition

* fix ruff complaint

* fix ruff check again

* avoid race condition when start() and stop() are called simultaneously from different threads

* avoid race condition in stop() when a job becomes active while shutting down

---------

Co-authored-by: Lincoln Stein <lstein@gmail.com>
Co-authored-by: Ryan Dick <ryanjdick3@gmail.com>
Co-authored-by: psychedelicious <4822129+psychedelicious@users.noreply.github.com>
Co-authored-by: Kent Keirsey <31807370+hipsterusername@users.noreply.github.com>
2023-12-22 12:35:57 -05:00
psychedelicious
78b29db458 feat(backend): disable graph library
The graph library occasionally causes issues when the default graph changes substantially between versions and pydantic validation fails. See #5289 for an example.

We are not currently using the graph library, so we can disable it until we are ready to use it. It's possible that the workflow library will supersede it anyways.
2023-12-23 00:04:48 +11:00
Lincoln Stein
1225c3fb47
addresses #5224 (#5332)
Co-authored-by: Lincoln Stein <lstein@gmail.com>
2023-12-22 12:30:51 +00:00
Brandon
32ad742f3e
Ti trigger from prompt util (#5294)
* Pull logic for extracting TI triggers into a util function

* Remove duplicate regex for ti triggers

* Fix linting for ruff

* Remove unused imports
2023-12-22 03:04:44 +00:00
psychedelicious
41cd40541a
Merge branch 'main' into feat/db/graceful-migrate-workflows 2023-12-22 12:21:52 +11:00
Millun Atluri
8dd55cc45e t2i with LoRA 2023-12-21 09:54:12 +11:00
psychedelicious
21ed2d42cd
Merge branch 'main' into feat/db/graceful-migrate-workflows 2023-12-20 21:54:21 +11:00
Millun Atluri
79cf3ec9a5 Add facedetailer workflow 2023-12-20 18:53:49 +11:00
Millun Atluri
37b76caccf Added default workflows 2023-12-20 17:42:14 +11:00
Lincoln Stein
fa3f1b6e41
[Feat] reimport model config records after schema migration (#5281)
* add code to repopulate model config records after schema update

* reformat for ruff

* migrate model records using db cursor rather than the ModelRecordConfigService

* ruff fixes

* tweak exception reporting

* fix: build frontend in  pypi-release workflow

This was missing, resulting in the 3.5.0rc1 having no frontend.

* fix: use node 18, set working directory

- Node 20 has  a problem with `pnpm`; set it to Node 18
- Set the working directory for the frontend commands

* Don't copy extraneous paths into installer .zip

* feat(installer): delete frontend build after creating installer

This prevents an empty `dist/` from breaking the app on startup.

* feat: add python dist as release artifact, as input to enable publish to pypi

- The release workflow never runs automatically. It must be manually kicked off.
- The release workflow has an input. When running it from the GH actions UI, you will see a "Publish build on PyPi" prompt. If this value is "true", the workflow will upload the build to PyPi, releasing it. If this is anything else (e.g. "false", the default), the workflow will build but not upload to PyPi.
- The `dist/` folder (where the python package is built) is uploaded as a workflow artifact as a zip file. This can be downloaded and inspected. This allows "dry" runs of the workflow.
- The workflow job and some steps have been renamed to clarify what they do

* translationBot(ui): update translation files

Updated by "Cleanup translation files" hook in Weblate.

Co-authored-by: Hosted Weblate <hosted@weblate.org>
Translate-URL: https://hosted.weblate.org/projects/invokeai/web-ui/
Translation: InvokeAI/Web UI

* freeze yaml migration logic at upgrade to 3.5

* moved migration code to migration_3

---------

Co-authored-by: Lincoln Stein <lstein@gmail.com>
Co-authored-by: psychedelicious <4822129+psychedelicious@users.noreply.github.com>
Co-authored-by: Hosted Weblate <hosted@weblate.org>
2023-12-19 17:01:47 -05:00
psychedelicious
45470a3ac8
Merge branch 'main' into feat/db/graceful-migrate-workflows 2023-12-18 23:32:28 +11:00
Lincoln Stein
09d1bc513d
Merge branch 'main' into refactor/model-manager2/mark-api-experimental 2023-12-18 07:04:00 -05:00
psychedelicious
b6ed4ba559 feat(db): handle PIL errors opening images gracefully
For example, if PIL tries to open a *really* big image, it will raise an exception to prevent reading a huge object into memory.
2023-12-18 18:02:31 +11:00
Lincoln Stein
74ea592d02 tag model manager v2 api as unstable 2023-12-17 14:16:45 -05:00
skunkworxdark
96a717c4ba In CalculateImageTilesEvenSplitInvocation to have overlap_fraction becomes just overlap. This is now in pixels rather than as a fraction of the tile size.
Update calc_tiles_even_split() with the same change. Ensuring Overlap is within allowed size

Update even_split tests
2023-12-17 15:10:50 +00:00
Lincoln Stein
72dca55e44 Merge branch 'feat/model_manager/search-by-format' of github.com:invoke-ai/InvokeAI into feat/model_manager/search-by-format 2023-12-14 23:55:08 -05:00
Lincoln Stein
264ea6d94d fix ruff errors 2023-12-14 23:54:59 -05:00
Lincoln Stein
60e3e653fa
Merge branch 'main' into feat/model_manager/search-by-format 2023-12-14 23:53:54 -05:00
Jonathan
ea4ef042f3 Ruff fixes 2023-12-14 12:47:10 +11:00
Jonathan
18b2bcbbee Added Classification from baseinvocation 2023-12-14 12:47:10 +11:00
Jonathan
5ad88c7f86 Fixed classification 2023-12-14 12:47:10 +11:00
Jonathan
3b04fef31d Added classification 2023-12-14 12:47:10 +11:00
Jonathan
bec888923a Fix for ruff 2023-12-14 12:47:10 +11:00
Jonathan
c6235049c7 Add an unsharp mask node to core nodes
Unsharp mask is an image operation that, despite its name, sharpens an image. Like a Gaussian blur, it takes a radius and strength.
2023-12-14 12:47:10 +11:00
psychedelicious
e10f6e8962 fix(nodes): mark CalculateImageTilesInvocation as beta
missed this when I added classification
2023-12-13 20:33:25 -05:00
psychedelicious
461e474394 fix(nodes): fix embedded workflows with IDs
This model was a bit too strict, and raised validation errors when workflows we expect to *not* have an ID (eg, an embedded workflow) have one.

Now it strips unknown attributes, allowing those workflows to load.
2023-12-14 11:38:04 +11:00
psychedelicious
f0c70fe3f1 fix(db): add error handling for workflow migration
- Handle an image file not existing despite being in the database.
- Add a simple pydantic model that tests only for the existence of a workflow's version.
- Check against this new model when migrating workflows, skipping if the workflow fails validation. If it succeeds, the frontend should be able to handle the workflow.
2023-12-14 10:16:56 +11:00
Lincoln Stein
569ae7c482 add ability to filter model listings by format 2023-12-13 15:59:21 -05:00
psychedelicious
f76b04a3b8 fix(db): rename "SQLiteMigrator" -> "SqliteMigrator" 2023-12-13 11:31:15 +11:00
psychedelicious
cc18d86f29
Merge branch 'main' into feat/db/migrations 2023-12-13 11:24:55 +11:00
psychedelicious
c50a49719b fix(db): raise a MigrationVersionError when invalid versions are used
This inherits from `ValueError`, so pydantic understands it when doing validation.
2023-12-13 11:21:16 +11:00
psychedelicious
ebf5f5d418 feat(db): address feedback, cleanup
- use simpler pattern for migration dependencies
- move SqliteDatabase & migration to utility method `init_db`, use this in both the app and tests, ensuring the same db schema is used in both
2023-12-13 11:19:59 +11:00
psychedelicious
d7cede6c28 chore/fix: bump fastapi to 0.105.0
This fixes a problem with `Annotated` which prevented us from using pydantic's `Field` to specify a discriminator for a union. We had to use FastAPI's `Body` as a workaround.
2023-12-13 09:48:34 +11:00
psychedelicious
2cdda1fda2 Merge remote-tracking branch 'origin/main' into feat/db/migrations 2023-12-12 17:22:52 +11:00
psychedelicious
3d64bc886d feat(nodes): flag all tiled upscaling nodes as beta 2023-12-12 16:43:05 +11:00
psychedelicious
1a136d6167 feat(nodes): fix classification docstrings 2023-12-12 16:43:05 +11:00
psychedelicious
43f2837117 feat(nodes): add invocation classifications
Invocations now have a classification:
- Stable: LTS
- Beta: LTS planned, API may change
- Prototype: No LTS planned, API may change, may be removed entirely

The `@invocation` decorator has a new arg `classification`, and an enum `Classification` is added to `baseinvocation.py`.

The default is Stable; this is a non-breaking change.

The classification is presented in the node header as a hammer icon (Beta) or flask icon (prototype).

The icon has a tooltip briefly describing the classification.
2023-12-12 16:43:05 +11:00
psychedelicious
5f77ef7e99 feat(db): improve docstrings in migrator 2023-12-12 16:30:57 +11:00
Lincoln Stein
d277bd3c38
Merge branch 'main' into feat/enhance-model-db-migrate-script 2023-12-12 00:24:43 -05:00
Kevin Turner
fd4e041e7c feat: serve HTTPS when configured with ssl_certfile 2023-12-12 16:01:43 +11:00
Lincoln Stein
15a3e8076f
Merge branch 'main' into feat/enhance-model-db-migrate-script 2023-12-11 23:10:04 -05:00
Lincoln Stein
b0cfa58526 allow the model record migrate script to update existing model records 2023-12-11 22:47:19 -05:00
psychedelicious
18ba7feca1 feat(db): update docstrings 2023-12-12 13:35:46 +11:00
psychedelicious
55b0c7cdc9 feat(db): tidy migration_2 2023-12-12 13:30:29 +11:00
psychedelicious
a69f518c76 feat(db): tidy dependencies for migrations 2023-12-12 13:09:09 +11:00
psychedelicious
0cf7fe43af feat(db): refactor migrate callbacks to use dependencies, remote pre/post callbacks 2023-12-12 12:35:42 +11:00
psychedelicious
6063760ce2 feat(db): tweak docstring 2023-12-12 11:13:40 +11:00
psychedelicious
c5ba4f2ea5 feat(db): remove file backups
Instead of mucking about with the filesystem, we rely on SQLite transactions to handle failed migrations.
2023-12-12 11:12:46 +11:00
psychedelicious
3414437eea feat(db): instantiate SqliteMigrator with a SqliteDatabase
Simplifies a couple things:
- Init is more straightforward
- It's clear in the migrator that the connection we are working with is related to the SqliteDatabase
2023-12-12 10:46:08 +11:00
psychedelicious
417db71471 feat(db): decouple SqliteDatabase from config object
- Simplify init args to path (None means use memory), logger, and verbose
- Add docstrings to SqliteDatabase (it had almost none)
- Update all usages of the class
2023-12-12 10:30:37 +11:00
psychedelicious
afe4e55bf9 feat(db): simplify migration registration validation
With the previous change to assert that the to_version == from_version + 1, this validation can be simpler.
2023-12-12 09:52:03 +11:00
psychedelicious
55acc16b2d feat(db): require migration versions to be consecutive 2023-12-12 09:43:09 +11:00
skunkworxdark
f762940335 Merge branch 'main' into tiled-upscaling-graph 2023-12-11 16:57:36 +00:00
skunkworxdark
fefb78795f - Even_spilt overlap renamed to overlap_fraction
- min_overlap removed * restrictions and round_to_8
- min_overlap handles tile size > image size by clipping the num tiles to 1.
- Updated assert test on min_overlap.
2023-12-11 16:55:27 +00:00
psychedelicious
290851016e feat(db): move sqlite_migrator into its own module 2023-12-11 16:41:30 +11:00
psychedelicious
fa7d002175 fix(tests): fix typing issues 2023-12-11 16:22:29 +11:00
psychedelicious@windows
f1b6f78319 fix(db): fix windows db migrator tests
- Ensure db files are closed before manipulating them
- Use contextlib.closing() so that sqlite connections are closed on existing the context
2023-12-11 16:14:25 +11:00
psychedelicious
4f3c32a2ee fix(db): remove errant print stmts 2023-12-11 16:14:25 +11:00
psychedelicious
41db92b9e8 feat(db): add check for missing migration from 0 2023-12-11 16:14:25 +11:00
psychedelicious
3227b30430 feat(db): extract non-stateful logic to class methods 2023-12-11 16:14:25 +11:00
psychedelicious
567f107a81 feat(db): return backup_db_path, move log stmt to run_migrations 2023-12-11 16:14:25 +11:00
psychedelicious
b3d5955bc7 fix(db): rename Migrator._migrations -> _migration_set 2023-12-11 16:14:25 +11:00
psychedelicious
8726b203d4 fix(db): fix migration chain validation 2023-12-11 16:14:25 +11:00
psychedelicious
b3f92e0547 fix(db): fix docstring 2023-12-11 16:14:25 +11:00
psychedelicious
72c9a7663f fix(db): add docstring 2023-12-11 16:14:25 +11:00
psychedelicious
fcb9e89bd7 feat(db): tidy db naming utils 2023-12-11 16:14:25 +11:00
psychedelicious
56966d6d05 feat(db): only reinit db if migrations occurred 2023-12-11 16:14:25 +11:00
psychedelicious
e46dc9b34e fix(db): close db conn before reinitializing 2023-12-11 16:14:25 +11:00
psychedelicious
e461f9925e feat(db): invert backup/restore logic
Do the migration on a temp copy of the db, then back up the original and move the temp into its file.
2023-12-11 16:14:25 +11:00
psychedelicious
abeb1bd3b3 feat(db): reduce power MigrateCallback, only gets cursor
use partial to provide extra dependencies for the image workflow migration function
2023-12-11 16:14:25 +11:00
psychedelicious
83e820d721 feat(db): decouple from SqliteDatabase 2023-12-11 16:14:25 +11:00
psychedelicious
f8e4b93a74 feat(db): add migration lock file 2023-12-11 16:14:25 +11:00
psychedelicious
0710ec30cf feat(db): incorporate feedback 2023-12-11 16:14:25 +11:00
psychedelicious
c382329e8c feat(db): move migrator out of SqliteDatabase 2023-12-11 16:14:25 +11:00
psychedelicious
f2c6819d68 feat(db): add SQLiteMigrator to perform db migrations 2023-12-11 16:14:25 +11:00
Lincoln Stein
fd68c47920
Merge branch 'main' into refactor/model-manager-3 2023-12-10 21:26:44 -05:00
psychedelicious
daf00efa4d fix(api): only attempt to serve UI build if it exists 2023-12-11 12:30:13 +11:00
Lincoln Stein
3b1ff4a7f4 resolve test failure caused by renamed sqlite_database module 2023-12-10 12:59:00 -05:00
Lincoln Stein
d7f7fbc8c2 Merge branch 'main' into refactor/model-manager-3 2023-12-10 12:55:28 -05:00
Lincoln Stein
2f3457c02a rename installer __del__() to stop(). Improve probe error messages 2023-12-10 12:55:01 -05:00
skunkworxdark
4c97b619fb Update tiles.py
merge with main
2023-12-09 22:05:23 +00:00
skunkworxdark
abdd840fb9 Merge branch 'main' into tiled-upscaling-graph 2023-12-09 22:03:18 +00:00
skunkworxdark
e656768eb2 more fixes from code review 2023-12-09 21:56:31 +00:00
skunkworxdark
494c2a9b05 Updates based on code review by @RyanJDick 2023-12-09 18:38:07 +00:00
psychedelicious
0ac33f36ef fix(tests): fix pydantic warning about deprecated fields
Calling `inspect.getmembers()` on a pydantic field results in `getattr` being called on all members of the field. Pydantic has some attrs that are marked deprecated.

In our test suite, we do not filter deprecation warnings, so this is surfaced.

Use a context manager to ignore deprecation warnings when calling the function.
2023-12-09 16:31:41 +11:00
psychedelicious
7436aa8e3a feat(workflow_records): do not use default_factory for workflow id
Using default_factory to autogenerate UUIDs doesn't make sense here, and results awkward typescript types.

Remove the default factory and instead manually create a UUID for workflow id. There are only two places where this needs to happen so it's not a big change.
2023-12-09 11:10:16 +11:00
psychedelicious
13c9f8ffb7 fix(nodes): fix mismatched invocation decorator
This got messed up during a merge commit
2023-12-09 11:10:16 +11:00
psychedelicious
c42d692ea6
feat: workflow library (#5148)
* chore: bump pydantic to 2.5.2

This release fixes pydantic/pydantic#8175 and allows us to use `JsonValue`

* fix(ui): exclude public/en.json from prettier config

* fix(workflow_records): fix SQLite workflow insertion to ignore duplicates

* feat(backend): update workflows handling

Update workflows handling for Workflow Library.

**Updated Workflow Storage**

"Embedded Workflows" are workflows associated with images, and are now only stored in the image files. "Library Workflows" are not associated with images, and are stored only in DB.

This works out nicely. We have always saved workflows to files, but recently began saving them to the DB in addition to in image files. When that happened, we stopped reading workflows from files, so all the workflows that only existed in images were inaccessible. With this change, access to those workflows is restored, and no workflows are lost.

**Updated Workflow Handling in Nodes**

Prior to this change, workflows were embedded in images by passing the whole workflow JSON to a special workflow field on a node. In the node's `invoke()` function, the node was able to access this workflow and save it with the image. This (inaccurately) models workflows as a property of an image and is rather awkward technically.

A workflow is now a property of a batch/session queue item. It is available in the InvocationContext and therefore available to all nodes during `invoke()`.

**Database Migrations**

Added a `SQLiteMigrator` class to handle database migrations. Migrations were needed to accomodate the DB-related changes in this PR. See the code for details.

The `images`, `workflows` and `session_queue` tables required migrations for this PR, and are using the new migrator. Other tables/services are still creating tables themselves. A followup PR will adapt them to use the migrator.

**Other/Support Changes**

- Add a `has_workflow` column to `images` table to indicate that the image has an embedded workflow.
- Add handling for retrieving the workflow from an image in python. The image file must be fetched, the workflow extracted, and then sent to client, avoiding needing the browser to parse the image file. With the `has_workflow` column, the UI knows if there is a workflow to be fetched, and only fetches when the user requests to load the workflow.
- Add route to get the workflow from an image
- Add CRUD service/routes for the library workflows
- `workflow_images` table and services removed (no longer needed now that embedded workflows are not in the DB)

* feat(ui): updated workflow handling (WIP)

Clientside updates for the backend workflow changes.

Includes roughed-out workflow library UI.

* feat: revert SQLiteMigrator class

Will pursue this in a separate PR.

* feat(nodes): do not overwrite custom node module names

Use a different, simpler method to detect if a node is custom.

* feat(nodes): restore WithWorkflow as no-op class

This class is deprecated and no longer needed. Set its workflow attr value to None (meaning it is now a no-op), and issue a warning when an invocation subclasses it.

* fix(nodes): fix get_workflow from queue item dict func

* feat(backend): add WorkflowRecordListItemDTO

This is the id, name, description, created at and updated at workflow columns/attrs. Used to display lists of workflowsl

* chore(ui): typegen

* feat(ui): add workflow loading, deleting to workflow library UI

* feat(ui): workflow library pagination button styles

* wip

* feat: workflow library WIP

- Save to library
- Duplicate
- Filter/sort
- UI/queries

* feat: workflow library - system graphs - wip

* feat(backend): sync system workflows to db

* fix: merge conflicts

* feat: simplify default workflows

- Rename "system" -> "default"
- Simplify syncing logic
- Update UI to match

* feat(workflows): update default workflows

- Update TextToImage_SD15
- Add TextToImage_SDXL
- Add README

* feat(ui): refine workflow list UI

* fix(workflow_records): typo

* fix(tests): fix tests

* feat(ui): clean up workflow library hooks

* fix(db): fix mis-ordered db cleanup step

It was happening before pruning queue items - should happen afterwards, else you have to restart the app again to free disk space made available by the pruning.

* feat(ui): tweak reset workflow editor translations

* feat(ui): split out workflow redux state

The `nodes` slice is a rather complicated slice. Removing `workflow` makes it a bit more reasonable.

Also helps to flatten state out a bit.

* docs: update default workflows README

* fix: tidy up unused files, unrelated changes

* fix(backend): revert unrelated service organisational changes

* feat(backend): workflow_records.get_many arg "filter_text" -> "query"

* feat(ui): use custom hook in current image buttons

Already in use elsewhere, forgot to use it here.

* fix(ui): remove commented out property

* fix(ui): fix workflow loading

- Different handling for loading from library vs external
- Fix bug where only nodes and edges loaded

* fix(ui): fix save/save-as workflow naming

* fix(ui): fix circular dependency

* fix(db): fix bug with releasing without lock in db.clean()

* fix(db): remove extraneous lock

* chore: bump ruff

* fix(workflow_records): default `category` to `WorkflowCategory.User`

This allows old workflows to validate when reading them from the db or image files.

* hide workflow library buttons if feature is disabled

---------

Co-authored-by: Mary Hipp <maryhipp@Marys-MacBook-Air.local>
2023-12-09 09:48:38 +11:00
Lincoln Stein
6e1e67aa72 remove source filtering from list_models() 2023-12-06 22:23:08 -05:00
skunkworxdark
674d9796d0 First check-in of new tile nodes
- calc_tiles_even_split
- calc_tiles_min_overlap
- merge_tiles_with_seam_blending
Update MergeTilesToImageInvocation with seam blending
2023-12-05 21:03:16 +00:00
Lincoln Stein
2d7555b7b8 Merge branch 'bugfix/log-levels' of github.com:invoke-ai/InvokeAI into bugfix/log-levels 2023-12-04 22:42:06 -05:00
Lincoln Stein
3c7d1fcd32 clean up get_logger() call 2023-12-04 22:41:59 -05:00
Lincoln Stein
3b06cc6782 reformatted using newer version of ruff 2023-12-04 21:15:56 -05:00
Lincoln Stein
fed2bf6dab Merge branch 'refactor/model-manager-3' of github.com:invoke-ai/InvokeAI into refactor/model-manager-3 2023-12-04 21:12:40 -05:00
Lincoln Stein
2b583ffcdf implement review suggestions from @RyanjDick 2023-12-04 21:12:10 -05:00
Lincoln Stein
6f46d15c05
Update invokeai/app/services/model_install/model_install_base.py
Co-authored-by: Ryan Dick <ryanjdick3@gmail.com>
2023-12-04 20:09:41 -05:00
Lincoln Stein
018ccebd6f make ModelLocalSource comparisons work across platforms 2023-12-04 19:07:25 -05:00
Lincoln Stein
620b2d477a implement suggestions from first review by @psychedelicious 2023-12-04 17:08:33 -05:00
Lincoln Stein
f73b678aae
Merge branch 'main' into refactor/model-manager-3 2023-12-04 17:06:36 -05:00
psychedelicious
0fdcc0af65 feat(nodes): add index and total to iterate output 2023-12-04 14:11:32 +11:00
Anthony Monthe
4fc2ed7195
Added full-version endpoint (#5206)
* Added get_app_deps endpoint

* Use importlib.version & added deps
2023-12-04 02:57:39 +00:00
psychedelicious
fb9b471150 feat(backend): move logic to clear latents to method 2023-12-01 17:44:07 -08:00
psychedelicious
3f0e0af177 feat(backend): only log pruned queue items / db freed space if > 0 2023-12-01 17:44:07 -08:00
psychedelicious
0228aba06f feat(backend): display freed space when cleaning DB 2023-12-01 17:44:07 -08:00
psychedelicious
1fd6666682 feat(backend): clear latents files on startup
Adds logic to `DiskLatentsStorage.start()` to empty the latents folder on startup.

Adds start and stop methods to `ForwardCacheLatentsStorage`. This is required for `DiskLatentsStorage.start()` to be called, due to how this particular service breaks the direct DI pattern, wrapping the underlying storage with a cache.
2023-12-01 17:44:07 -08:00
Lincoln Stein
778fd55f0d Merge branch 'main' into refactor/model-manager-3 2023-12-01 09:15:18 -05:00
Ryan Dick
984e609c61 (minor) Tweak field ordering and field names for tiling nodes. 2023-11-30 07:53:27 -08:00
Ryan Dick
57e70aaf50 Change input field ordering of CropLatentsCoreInvocation to match ImageCropInvocation. 2023-11-30 07:53:27 -08:00
Ryan Dick
32da359ba5 Infer a tight-fitting output image size from the passed tiles in MergeTilesToImageInvocation. 2023-11-30 07:53:27 -08:00
Ryan Dick
b19ed36b43 Add width and height fields to TileToPropertiesInvocation output to avoid having to calculate them with math nodes. 2023-11-30 07:53:27 -08:00
Ryan Dick
e5a212b5c8 Update tiling nodes to use width-before-height field ordering convention. 2023-11-30 07:53:27 -08:00
Ryan Dick
9b863fb9bc Rename CropLatentsInvocation -> CropLatentsCoreInvocation to prevent conflict with custom node. And other minor tidying. 2023-11-30 07:53:27 -08:00
Ryan Dick
7cab51745b Improve documentation of CropLatentsInvocation. 2023-11-30 07:53:27 -08:00
Ryan Dick
18c6ff427e Use LATENT_SCALE_FACTOR = 8 constant in CropLatentsInvocation. 2023-11-30 07:53:27 -08:00
Ryan Dick
843f2d71d6 Copy CropLatentsInvocation from 74647fa9c1/images_to_grids.py (L1117C1-L1167C80). 2023-11-30 07:53:27 -08:00
Ryan Dick
67540c9ee0 (minor) Add 'Invocation' suffix to all tiling node classes. 2023-11-30 07:53:27 -08:00
Ryan Dick
7f816c9243 Tidy up tiles invocations, add documentation. 2023-11-30 07:53:27 -08:00
Ryan Dick
29eade4880 Add nodes for tile splitting and merging. The main motivation for these nodes is for use in tiled upscaling workflows. 2023-11-30 07:53:27 -08:00
ymgenesis
3e01c396e1
CenterPadCrop node (#3861)
* add centerpadcrop node

- Allows users to add padding to or crop images from the center
- Also outputs a white mask with the dimensions of the output image for use with outpainting

* add CenterPadCrop to NODES.md

Updates NODES.md with CenterPadCrop entry.

* remove mask & output class

- Remove "ImageMaskOutput" where both image and mask are output
- Remove ability to output mask from node

---------

Co-authored-by: psychedelicious <4822129+psychedelicious@users.noreply.github.com>
2023-11-30 21:15:59 +11:00
Damian Stewart
0beb08686c
Add CFG Rescale option for supporting zero-terminal SNR models (#4335)
* add support for CFG rescale

* fix typo

* move rescale position and tweak docs

* move input position

* implement suggestions from github and discord

* cleanup unused code

* add back dropped FieldDescription

* fix(ui): revert unrelated UI changes

* chore(nodes): bump denoise_latents version 1.4.0 -> 1.5.0

* feat(nodes): add cfg_rescale_multiplier to metadata node

* feat(ui): add cfg rescale multiplier to linear UI

- add param to state
- update graph builders
- add UI under advanced
- add metadata handling & recall
- regen types

* chore: black

* fix(backend): make `StableDiffusionGeneratorPipeline._rescale_cfg()` staticmethod

This doesn't need access to class.

* feat(backend): add docstring for `_rescale_cfg()` method

* feat(ui): update cfg rescale mult translation string

---------

Co-authored-by: psychedelicious <4822129+psychedelicious@users.noreply.github.com>
2023-11-30 20:55:20 +11:00
skunkworxdark
77933a0a85 Update prompt.py
bumped version to 1.0.1
2023-11-29 23:40:10 +11:00
skunkworxdark
2a087bf161 Update prompt.py
Use UTF-8 encoding on reading prompts from files to allow Unicode characters to load correctly. 
The following examples currently will not load correctly from a file:

Hello, 世界!
😭🤮 💔
2023-11-29 23:40:10 +11:00
psychedelicious
4468581d2e fix(nodes): remove extraneous del 2023-11-29 10:49:31 +11:00
psychedelicious
a1705dc6b3 fix(nodes): fix loading node pack display 2023-11-29 10:49:31 +11:00
psychedelicious
4af4486dd9 feat(nodes,ui): add detection of custom nodes
Custom nodes have a new attribute `node_pack` indicating the node pack they came from.

- This is displayed in the UI in the icon icon tooltip.
- If a workflow is loaded and a node is unavailable, its node pack will be displayed (if it is known).
- If a workflow is migrated from v1 to v2, and the node is unknown, it falls back to "Unknown". If the missing node pack is installed and the node is updated, the node pack will be updated as expected.
2023-11-29 10:49:31 +11:00
psychedelicious
514c49d946 feat(nodes): warn if node has no version specified; fall back on 1.0.0 2023-11-29 10:49:31 +11:00
psychedelicious
858bcdd3ff feat(nodes): improve docstrings in baseinvocation, disambiguate method names 2023-11-29 10:49:31 +11:00
psychedelicious
86a74e929a feat(ui): add support for custom field types
Node authors may now create their own arbitrary/custom field types. Any pydantic model is supported.

Two notes:
1. Your field type's class name must be unique.

Suggest prefixing fields with something related to the node pack as a kind of namespace.

2. Custom field types function as connection-only fields.

For example, if your custom field has string attributes, you will not get a text input for that attribute when you give a node a field with your custom type.

This is the same behaviour as other complex fields that don't have custom UIs in the workflow editor - like, say, a string collection.

feat(ui): fix tooltips for custom types

We need to hold onto the original type of the field so they don't all just show up as "Unknown".

fix(ui): fix ts error with custom fields

feat(ui): custom field types connection validation

In the initial commit, a custom field's original type was added to the *field templates* only as `originalType`. Custom fields' `type` property was `"Custom"`*. This allowed for type safety throughout the UI logic.

*Actually, it was `"Unknown"`, but I changed it to custom for clarity.

Connection validation logic, however, uses the *field instance* of the node/field. Like the templates, *field instances* with custom types have their `type` set to `"Custom"`, but they didn't have an `originalType` property. As a result, all custom fields could be connected to all other custom fields.

To resolve this, we need to add `originalType` to the *field instances*, then switch the validation logic to use this instead of `type`.

This ended up needing a bit of fanagling:

- If we make `originalType` a required property on field instances, existing workflows will break during connection validation, because they won't have this property. We'd need a new layer of logic to migrate the workflows, adding the new `originalType` property.

While this layer is probably needed anyways, typing `originalType` as optional is much simpler. Workflow migration logic can come layer.

(Technically, we could remove all references to field types from the workflow files, and let the templates hold all this information. This feels like a significant change and I'm reluctant to do it now.)

- Because `originalType` is optional, anywhere we care about the type of a field, we need to use it over `type`. So there are a number of `field.originalType ?? field.type` expressions. This is a bit of a gotcha, we'll need to remember this in the future.

- We use `Array.prototype.includes()` often in the workflow editor, e.g. `COLLECTION_TYPES.includes(type)`. In these cases, the const array is of type `FieldType[]`, and `type` is is `FieldType`.

Because we now support custom types, the arg `type` is now widened from `FieldType` to `string`.

This causes a TS error. This behaviour is somewhat controversial (see https://github.com/microsoft/TypeScript/issues/14520). These expressions are now rewritten as `COLLECTION_TYPES.some((t) => t === type)` to satisfy TS. It's logically equivalent.

fix(ui): typo

feat(ui): add CustomCollection and CustomPolymorphic field types

feat(ui): add validation for CustomCollection & CustomPolymorphic types

- Update connection validation for custom types
- Use simple string parsing to determine if a field is a collection or polymorphic type.
- No longer need to keep a list of collection and polymorphic types.
- Added runtime checks in `baseinvocation.py` to ensure no fields are named in such a way that it could mess up the new parsing

chore(ui): remove errant console.log

fix(ui): rename 'nodes.currentConnectionFieldType' -> 'nodes.connectionStartFieldType'

This was confusingly named and kept tripping me up. Renamed to be consistent with the `reactflow` `ConnectionStartParams` type.

fix(ui): fix ts error

feat(nodes): add runtime check for custom field names

"Custom", "CustomCollection" and "CustomPolymorphic" are reserved field names.

chore(ui): add TODO for revising field type names

wip refactor fieldtype structured

wip refactor field types

wip refactor types

wip refactor types

fix node layout

refactor field types

chore: mypy

organisation

organisation

organisation

fix(nodes): fix field orig_required, field_kind and input statuses

feat(nodes): remove broken implementation of default_factory on InputField

Use of this could break connection validation due to the difference in node schemas required fields and invoke() required args.

Removed entirely for now. It wasn't ever actually used by the system, because all graphs always had values provided for fields where default_factory was used.

Also, pydantic is smart enough to not reuse the same object when specifying a default value - it clones the object first. So, the common pattern of `default_factory=list` is extraneous. It can just be `default=[]`.

fix(nodes): fix InputField name validation

workflow validation

validation

chore: ruff

feat(nodes): fix up baseinvocation comments

fix(ui): improve typing & logic of buildFieldInputTemplate

improved error handling in parseFieldType

fix: back compat for deprecated default_factory and UIType

feat(nodes): do not show node packs loaded log if none loaded

chore(ui): typegen
2023-11-29 10:49:31 +11:00
Lincoln Stein
ecd3dcd5df
Merge branch 'main' into refactor/model-manager-3 2023-11-27 22:15:51 -05:00
psychedelicious
e28262ebd9 fix(config): use public import path for JsonDict 2023-11-28 09:30:49 +11:00
Lincoln Stein
250ee4b11c resolve which paths can be None 2023-11-28 09:30:49 +11:00
Lincoln Stein
b7293d638b fix import block ordering 2023-11-28 09:30:49 +11:00
Lincoln Stein
eee863e380 fix type mismatches in invokeai.app.services.config.config_base & config_default 2023-11-28 09:30:49 +11:00
Lincoln Stein
a79e814c8d
Merge branch 'main' into refactor/model-manager-3 2023-11-27 16:06:42 -05:00
psychedelicious
1d8f44d356 fix(backend): remove inaccurate comments in upscale.py 2023-11-28 07:58:22 +11:00
psychedelicious
7653d21cf5 feat(backend): rename realesrgan class & upscale method 2023-11-28 07:58:22 +11:00
psychedelicious
46a2d83b84 feat(backend): organise realesrgan code, add license
- Moved util to own folder
- BSD3 License for RealESRGAN repo added
2023-11-28 07:58:22 +11:00
psychedelicious
2192210910 feat(nodes): remove dependency on realesrgan
We used the `RealESRGANer` utility class from the repo. It handled model loading and tiled upscaling logic.

Unfortunately, it hasn't been updated in over a year, had no types, and annoyingly printed to console.

I've adapted the class, cleaning it up a bit and removing the bits that are not relevant for us.

Upscaling functionality is identical.
2023-11-28 07:58:22 +11:00
Lincoln Stein
8ef596eac7 further changes for ruff 2023-11-26 17:13:31 -05:00
Lincoln Stein
8695ad6f59 all features implemented, docs updated, ready for review 2023-11-26 13:18:21 -05:00
Lincoln Stein
dc5c452ef9 rename test/nodes to test/aa_nodes to ensure these tests run first 2023-11-26 09:38:30 -05:00
Lincoln Stein
8aefe2cefe import_model and list_install_jobs router APIs written 2023-11-25 21:45:59 -05:00
Lincoln Stein
ec510d34b5 fix model probing for controlnet checkpoint legacy config files 2023-11-25 15:53:22 -05:00
Lincoln Stein
19baea1883 all backend features in place; config scanning is failing on controlnet 2023-11-24 19:37:46 -05:00
Lincoln Stein
80bc9be3ab make install_path and register_path work; refactor model probing 2023-11-23 23:15:32 -05:00
Lincoln Stein
4aab728590 move name/description logic into model_probe.py 2023-11-22 22:29:02 -05:00
Lincoln Stein
9cf060115d Merge branch 'main' into refactor/model-manager-3 2023-11-22 22:28:31 -05:00
psychedelicious
da443973cb chore: ruff 2023-11-21 20:22:27 +11:00
Lincoln Stein
9ea3126118 start implementation of installer 2023-11-20 23:02:30 -05:00
Lincoln Stein
6c56233edc define install abstract base class 2023-11-20 21:57:10 -05:00
Ryan Dick
d756c9b10a Fix double LoRA patching of the UNet. This was presumably added by accident due to a previous merge conflict. 2023-11-17 12:05:04 -08:00
psychedelicious
91ef24e15c fix(nodes,ui): fix missed/canvas temp images in gallery
Resolves two bugs introduced in #5106:

1. Linear UI images sometimes didn't make it to the gallery.

This was a race condition. The VAE decode nodes were handled by the socketInvocationComplete listener. At that moment, the image was marked as intermediate. Immediately after this node was handled, a LinearUIOutputInvocation, introduced in #5106, was handled by socketInvocationComplete. This node internally sets changed the image to not intermediate.

During the handling of that socketInvocationComplete, RTK Query would sometimes use its cache instead of retrieving the image DTO again. The result is that the UI never got the message that the image was not intermediate, so it wasn't added to the gallery.

This is resolved by refactoring the socketInvocationComplete listener. We now skip the gallery processing for linear UI events, except for the LinearUIOutputInvocation. Images now always make it to the gallery, and network requests to get image DTOs are substantially reduced.

2. Canvas temp images always went into the gallery

The LinearUIOutputInvocation was always setting its image's is_intermediate to false. This included all canvas images and resulted in all canvas temp images going to gallery.

This is resolved by making LinearUIOutputInvocation set is_intermediate based on `self.is_intermediate`. The behaviour now more or less mirroring the behaviour of is_intermediate on other image-outputting nodes, except it doesn't save the image again - only changes it.

One extra minor change - LinearUIOutputInvocation only changes is_intermediate if it differs from the image's current setting. Very minor optimisation.
2023-11-17 07:32:04 +11:00
psychedelicious
4599517c6c feat: add private node for linear UI image outputting
Add a LinearUIOutputInvocation node to be the new terminal node for Linear UI graphs. This node is private and hidden from the Workflow Editor, as it is an implementation detail.

The Linear UI was using the Save Image node for this purpose. It allowed every linear graph to end a single node type, which handled saving metadata and board. This substantially reduced the complexity of the linear graphs.

This caused two related issues:
- Images were saved to disk twice
- Noticeable delay between when an image was decoded and showed up in the UI

To resolve this, the new LinearUIOutputInvocation node will handle adding an image to a board if one is provided.

Metadata is no longer provided in this unified node. Instead, the metadata graph helpers now need to know the node to add metadata to and provide it to the last node that actually outputs an image. This is a `l2i` node for txt2img & img2img graphs, and a different image-outputting node for canvas graphs.

HRF poses another complication, in that it changes the terminal node. To handle this, a new metadata util is added called `setMetadataReceivingNode()`. HRF calls this to change the node that should receive the graph's metadata.

This resolves the duplicate images issue and improves perf without otherwise changing the user experience.
2023-11-16 18:56:59 +11:00
psychedelicious
cc747c066c fix(nodes): fix hrf_enabled metadata item
It was a float but should be a bool
2023-11-16 18:47:31 +11:00
psychedelicious
5cb3fdb64c fix(nodes): bump version of nodes post-pydantic v2 2023-11-16 11:14:26 +11:00
psychedelicious
e8b83fecff fix(backend): apply clip skip after lora
This handles LoRAs that attempt to modify layers skipped by CLIP Skip.
2023-11-14 11:30:15 +11:00
Lincoln Stein
acc0a29dca fixed ruff formatting issues 2023-11-13 18:15:17 -05:00
Lincoln Stein
38c1436f02 resolve conflicts; blackify 2023-11-13 18:12:45 -05:00
Lincoln Stein
efbdb75568 implement psychedelicious recommendations as of 13 November 2023-11-13 17:05:01 -05:00
psychedelicious
428f0b265f feat(api): add log stmt to update_model_record route 2023-11-14 08:06:35 +11:00
psychedelicious
7daee41ad2 fix(api): remove unused ModelsListValidator 2023-11-14 08:01:44 +11:00
psychedelicious
7cdd7b6ad7 feat(api): simplifiy list_model_records handler 2023-11-14 08:00:21 +11:00
psychedelicious
bc64cde6f9 chore: ruff lint 2023-11-14 07:57:07 +11:00
psychedelicious
4465f97cdf
Merge branch 'main' into refactor/model-manager-2 2023-11-14 07:51:57 +11:00
Lincoln Stein
8afe517204 add note about discriminated union and Body() issue; blackified 2023-11-12 16:50:05 -05:00
Lincoln Stein
ef8dcf5fae blackify 2023-11-12 14:20:32 -05:00
Lincoln Stein
024a156114 isort 2023-11-11 13:58:36 -05:00
Lincoln Stein
af2264b6eb implement workaround for FastAPI and discriminated unions in Body parameter 2023-11-11 12:22:38 -05:00
psychedelicious
520ccdb0a9
Merge branch 'main' into feat/ruff 2023-11-11 15:07:35 +11:00
Lincoln Stein
2b36565e9e awkward workaround for double-Annotated in model_record route 2023-11-10 21:32:44 -05:00
Lincoln Stein
f2c3b7c317 Merge branch 'refactor/model-manager-2' of github.com:invoke-ai/InvokeAI into refactor/model-manager-2 2023-11-10 19:47:01 -05:00
Lincoln Stein
67751a01ab remove unused import 2023-11-10 19:25:05 -05:00
Lincoln Stein
cb8cdefd59
Merge branch 'main' into refactor/model-manager-2 2023-11-10 19:24:19 -05:00
Lincoln Stein
3a6ba236f5 replace _class_map in ModelConfigFactory with a nested discriminated union 2023-11-10 19:14:15 -05:00
Paul Curry
1c7ea57492
feat (ui, generation): High Resolution Fix- added automatic resolution toggle and replaced latent upscale with two improved methods (#4905)
* working

* added selector for method

* refactoring graph

* added ersgan method

* fixing yarn build

* add tooltips

* a conjuction

* rephrase

* removed manual sliders, set HRF to calculate dimensions automatically to match 512^2 pixels

* working

* working

* working

* fixed tooltip

* add hrf to use all parameters

* adding hrf method to parameters

* working on parameter recall

* working on parameter recall

* cleaning

* fix(ui): fix unnecessary casts in addHrfToGraph

* chore(ui): use camelCase in addHrfToGraph

* fix(ui): do not add HRF metadata unless HRF is added to graph

* fix(ui): remove unused imports in addHrfToGraph

* feat(ui): do not hide HRF params when disabled, only disable them

* fix(ui): remove unused vars in addHrfToGraph

* feat(ui): default HRF str to 0.35, method ESRGAN

* fix(ui): use isValidBoolean to check hrfEnabled param

* fix(nodes): update CoreMetadataInvocation fields for HRF

* feat(ui): set hrf strength default to 0.45

* fix(ui): set default hrf strength in configSlice

* feat(ui): use translations for HRF features

---------

Co-authored-by: psychedelicious <4822129+psychedelicious@users.noreply.github.com>
2023-11-11 00:11:46 +00:00
psychedelicious
6494e8e551 chore: ruff format 2023-11-11 10:55:40 +11:00
psychedelicious
513fceac82 chore: ruff check - fix pycodestyle 2023-11-11 10:55:33 +11:00
psychedelicious
99a8ebe3a0 chore: ruff check - fix flake8-bugbear 2023-11-11 10:55:28 +11:00
psychedelicious
3a136420d5 chore: ruff check - fix flake8-comprensions 2023-11-11 10:55:23 +11:00
Lincoln Stein
bd56e9bc81 remove cruft code from router 2023-11-10 18:49:25 -05:00
Lincoln Stein
b55fc2935e resolve conflicts with commits done on github 2023-11-10 18:26:48 -05:00
Lincoln Stein
0544917161 multiple small fixes suggested in reviews from psychedelicious and ryan 2023-11-10 18:25:37 -05:00
Lincoln Stein
1161dfe055
Update invokeai/app/api/routers/model_records.py
Co-authored-by: Ryan Dick <ryanjdick3@gmail.com>
2023-11-10 18:24:55 -05:00
Lincoln Stein
433f347d7e
Update invokeai/app/api/routers/model_records.py
Co-authored-by: Ryan Dick <ryanjdick3@gmail.com>
2023-11-10 18:22:54 -05:00
Lincoln Stein
9316534d97
Update invokeai/app/services/model_records/model_records_sql.py
Co-authored-by: psychedelicious <4822129+psychedelicious@users.noreply.github.com>
2023-11-10 17:58:15 -05:00
Lincoln Stein
f1c195afb7 Merge branch 'main' into refactor/model-manager-2 2023-11-10 17:54:28 -05:00
Brandon Rising
41f7aa6ab4 Remove unused import: 2023-11-09 15:06:01 -05:00
Brandon Rising
9bec755198 Upstream diffusers PR was merged, this no longer seems necessary 2023-11-09 15:02:24 -05:00
psychedelicious
6aa87f973e fix(nodes): create app/shared/ module to prevent circular imports
We have a number of shared classes, objects, and functions that are used in multiple places. This causes circular import issues.

This commit creates a new `app/shared/` module to hold these shared classes, objects, and functions.

Initially, only `FreeUConfig` and `FieldDescriptions` are moved here. This resolves a circular import issue with custom nodes.

Other shared classes, objects, and functions will be moved here in future commits.
2023-11-09 16:41:55 +11:00
Kevin Turner
f793fdf3d4 fix(socketio): leave room on unsubscribe
https://discord.com/channels/1020123559063990373/1049495067846524939/1171976251704086559
2023-11-09 12:12:32 +11:00
Lincoln Stein
3b363d0258 fix flake8 lint check failures 2023-11-08 16:52:46 -05:00
Lincoln Stein
36e0faea6b blackify 2023-11-08 16:47:03 -05:00
Lincoln Stein
927f8a66e6 Merge branch 'main' into refactor/model-manager-2 2023-11-08 16:46:08 -05:00
Lincoln Stein
eebc0e7315 Merge branch 'refactor/model-manager-2' of github.com:invoke-ai/InvokeAI into refactor/model-manager-2 2023-11-08 16:45:29 -05:00
Lincoln Stein
6b173cc66f multiple small stylistic changes requested by reviewers 2023-11-08 16:45:26 -05:00
Lincoln Stein
b4732a7308
Update invokeai/app/services/model_records/model_records_base.py
Co-authored-by: Ryan Dick <ryanjdick3@gmail.com>
2023-11-08 13:50:40 -05:00
Lincoln Stein
344a56327a
Update invokeai/app/services/model_records/model_records_base.py
Co-authored-by: Ryan Dick <ryanjdick3@gmail.com>
2023-11-08 13:50:01 -05:00
Millun Atluri
4cfd55936c run black formatting 2023-11-07 16:06:18 +11:00
Millun Atluri
5c3a27aac6 fixed sorts 2023-11-07 16:03:06 +11:00
Millun Atluri
d573a23090 Moved FreeU Config Import 2023-11-07 15:48:53 +11:00
Lincoln Stein
ce22c0fbaa sync pydantic and sql field names; merge routes 2023-11-06 18:08:57 -05:00
Kent Keirsey
ff8a8a1963
Merge branch 'main' into feat/nodes/freeu 2023-11-06 09:04:54 -08:00
Kent Keirsey
f8f1740668 Set Defaults to 1 2023-11-06 07:11:16 -08:00
Kent Keirsey
e66d0f7372
Merge branch 'main' into feat/nodes/freeu 2023-11-06 05:39:58 -08:00
Lincoln Stein
2d051559d1 fix flake8 complaints 2023-11-05 21:45:08 -05:00
Lincoln Stein
db9cef0092 re-run isort 2023-11-04 23:50:07 -04:00
Lincoln Stein
72c34aea75 added add_model_record and get_model_record to router api 2023-11-04 23:42:44 -04:00
Lincoln Stein
edeea5237b add sql-based model config store and api 2023-11-04 23:03:26 -04:00
Ryan Dick
6e7a3f0546 (minor) Fix static checks and typo. 2023-11-02 19:20:37 -07:00
Ryan Dick
4a683cc669 Add a app config parameter to control the ModelCache logging behavior. 2023-11-02 19:20:37 -07:00
Ryan Dick
379d68f595 Patch LoRA on device when model is already on device. 2023-11-02 10:03:17 -07:00
Kent Keirsey
bb68175fd0 Add negative IP Adapter support 2023-10-31 14:30:24 +11:00
psychedelicious
03a64275c6 fix(db): fix deprecated pydantic .json() method 2023-10-31 04:34:51 +11:00
psychedelicious
55bfadfd0b fix(nodes): fix DenoiseMaskField.masked_latents_name
This optional field needs to have a default of `None`.
2023-10-31 04:18:09 +11:00
psychedelicious
859e3d5a61 chore: flake8 2023-10-30 01:49:10 +11:00
Lincoln Stein
3546c41f4a close #4975 2023-10-23 18:48:14 -04:00
psychedelicious
8615d53e65 fix(nodes): fix missing generation modes
Lax typing on the metadata util functions allowed a typing issue to slip through. Fixed the lax typing, updated core metadata node.
2023-10-20 16:22:56 +11:00
psychedelicious
824702de99 feat(nodes): change expected structure for custom nodes 2023-10-20 14:28:16 +11:00
psychedelicious
8604943e89 feat(nodes): simple custom nodes
Custom nodes may be places in `$INVOKEAI_ROOT/nodes/` (configurable with `custom_nodes_dir` option).

On app startup, an `__init__.py` is copied into the custom nodes dir, which recursively loads all python files in the directory as modules (files starting with `_` are ignored). The custom nodes dir is now a python module itself.

When we `from invocations import *` to load init all invocations, we load the custom nodes dir, registering all custom nodes.
2023-10-20 14:28:16 +11:00
psychedelicious
dcd11327c1 fix(db): remove unused, commented out methods 2023-10-20 12:05:13 +11:00