Commit Graph

1020 Commits

Author SHA1 Message Date
21ed2d42cd Merge branch 'main' into feat/db/graceful-migrate-workflows 2023-12-20 21:54:21 +11:00
79cf3ec9a5 Add facedetailer workflow 2023-12-20 18:53:49 +11:00
37b76caccf Added default workflows 2023-12-20 17:42:14 +11:00
fa3f1b6e41 [Feat] reimport model config records after schema migration (#5281)
* add code to repopulate model config records after schema update

* reformat for ruff

* migrate model records using db cursor rather than the ModelRecordConfigService

* ruff fixes

* tweak exception reporting

* fix: build frontend in  pypi-release workflow

This was missing, resulting in the 3.5.0rc1 having no frontend.

* fix: use node 18, set working directory

- Node 20 has  a problem with `pnpm`; set it to Node 18
- Set the working directory for the frontend commands

* Don't copy extraneous paths into installer .zip

* feat(installer): delete frontend build after creating installer

This prevents an empty `dist/` from breaking the app on startup.

* feat: add python dist as release artifact, as input to enable publish to pypi

- The release workflow never runs automatically. It must be manually kicked off.
- The release workflow has an input. When running it from the GH actions UI, you will see a "Publish build on PyPi" prompt. If this value is "true", the workflow will upload the build to PyPi, releasing it. If this is anything else (e.g. "false", the default), the workflow will build but not upload to PyPi.
- The `dist/` folder (where the python package is built) is uploaded as a workflow artifact as a zip file. This can be downloaded and inspected. This allows "dry" runs of the workflow.
- The workflow job and some steps have been renamed to clarify what they do

* translationBot(ui): update translation files

Updated by "Cleanup translation files" hook in Weblate.

Co-authored-by: Hosted Weblate <hosted@weblate.org>
Translate-URL: https://hosted.weblate.org/projects/invokeai/web-ui/
Translation: InvokeAI/Web UI

* freeze yaml migration logic at upgrade to 3.5

* moved migration code to migration_3

---------

Co-authored-by: Lincoln Stein <lstein@gmail.com>
Co-authored-by: psychedelicious <4822129+psychedelicious@users.noreply.github.com>
Co-authored-by: Hosted Weblate <hosted@weblate.org>
2023-12-19 17:01:47 -05:00
b6ed4ba559 feat(db): handle PIL errors opening images gracefully
For example, if PIL tries to open a *really* big image, it will raise an exception to prevent reading a huge object into memory.
2023-12-18 18:02:31 +11:00
60e3e653fa Merge branch 'main' into feat/model_manager/search-by-format 2023-12-14 23:53:54 -05:00
461e474394 fix(nodes): fix embedded workflows with IDs
This model was a bit too strict, and raised validation errors when workflows we expect to *not* have an ID (eg, an embedded workflow) have one.

Now it strips unknown attributes, allowing those workflows to load.
2023-12-14 11:38:04 +11:00
f0c70fe3f1 fix(db): add error handling for workflow migration
- Handle an image file not existing despite being in the database.
- Add a simple pydantic model that tests only for the existence of a workflow's version.
- Check against this new model when migrating workflows, skipping if the workflow fails validation. If it succeeds, the frontend should be able to handle the workflow.
2023-12-14 10:16:56 +11:00
569ae7c482 add ability to filter model listings by format 2023-12-13 15:59:21 -05:00
f76b04a3b8 fix(db): rename "SQLiteMigrator" -> "SqliteMigrator" 2023-12-13 11:31:15 +11:00
cc18d86f29 Merge branch 'main' into feat/db/migrations 2023-12-13 11:24:55 +11:00
c50a49719b fix(db): raise a MigrationVersionError when invalid versions are used
This inherits from `ValueError`, so pydantic understands it when doing validation.
2023-12-13 11:21:16 +11:00
ebf5f5d418 feat(db): address feedback, cleanup
- use simpler pattern for migration dependencies
- move SqliteDatabase & migration to utility method `init_db`, use this in both the app and tests, ensuring the same db schema is used in both
2023-12-13 11:19:59 +11:00
d7cede6c28 chore/fix: bump fastapi to 0.105.0
This fixes a problem with `Annotated` which prevented us from using pydantic's `Field` to specify a discriminator for a union. We had to use FastAPI's `Body` as a workaround.
2023-12-13 09:48:34 +11:00
2cdda1fda2 Merge remote-tracking branch 'origin/main' into feat/db/migrations 2023-12-12 17:22:52 +11:00
5f77ef7e99 feat(db): improve docstrings in migrator 2023-12-12 16:30:57 +11:00
d277bd3c38 Merge branch 'main' into feat/enhance-model-db-migrate-script 2023-12-12 00:24:43 -05:00
fd4e041e7c feat: serve HTTPS when configured with ssl_certfile 2023-12-12 16:01:43 +11:00
b0cfa58526 allow the model record migrate script to update existing model records 2023-12-11 22:47:19 -05:00
18ba7feca1 feat(db): update docstrings 2023-12-12 13:35:46 +11:00
55b0c7cdc9 feat(db): tidy migration_2 2023-12-12 13:30:29 +11:00
a69f518c76 feat(db): tidy dependencies for migrations 2023-12-12 13:09:09 +11:00
0cf7fe43af feat(db): refactor migrate callbacks to use dependencies, remote pre/post callbacks 2023-12-12 12:35:42 +11:00
6063760ce2 feat(db): tweak docstring 2023-12-12 11:13:40 +11:00
c5ba4f2ea5 feat(db): remove file backups
Instead of mucking about with the filesystem, we rely on SQLite transactions to handle failed migrations.
2023-12-12 11:12:46 +11:00
3414437eea feat(db): instantiate SqliteMigrator with a SqliteDatabase
Simplifies a couple things:
- Init is more straightforward
- It's clear in the migrator that the connection we are working with is related to the SqliteDatabase
2023-12-12 10:46:08 +11:00
417db71471 feat(db): decouple SqliteDatabase from config object
- Simplify init args to path (None means use memory), logger, and verbose
- Add docstrings to SqliteDatabase (it had almost none)
- Update all usages of the class
2023-12-12 10:30:37 +11:00
afe4e55bf9 feat(db): simplify migration registration validation
With the previous change to assert that the to_version == from_version + 1, this validation can be simpler.
2023-12-12 09:52:03 +11:00
55acc16b2d feat(db): require migration versions to be consecutive 2023-12-12 09:43:09 +11:00
290851016e feat(db): move sqlite_migrator into its own module 2023-12-11 16:41:30 +11:00
fa7d002175 fix(tests): fix typing issues 2023-12-11 16:22:29 +11:00
f1b6f78319 fix(db): fix windows db migrator tests
- Ensure db files are closed before manipulating them
- Use contextlib.closing() so that sqlite connections are closed on existing the context
2023-12-11 16:14:25 +11:00
4f3c32a2ee fix(db): remove errant print stmts 2023-12-11 16:14:25 +11:00
41db92b9e8 feat(db): add check for missing migration from 0 2023-12-11 16:14:25 +11:00
3227b30430 feat(db): extract non-stateful logic to class methods 2023-12-11 16:14:25 +11:00
567f107a81 feat(db): return backup_db_path, move log stmt to run_migrations 2023-12-11 16:14:25 +11:00
b3d5955bc7 fix(db): rename Migrator._migrations -> _migration_set 2023-12-11 16:14:25 +11:00
8726b203d4 fix(db): fix migration chain validation 2023-12-11 16:14:25 +11:00
b3f92e0547 fix(db): fix docstring 2023-12-11 16:14:25 +11:00
72c9a7663f fix(db): add docstring 2023-12-11 16:14:25 +11:00
fcb9e89bd7 feat(db): tidy db naming utils 2023-12-11 16:14:25 +11:00
56966d6d05 feat(db): only reinit db if migrations occurred 2023-12-11 16:14:25 +11:00
e46dc9b34e fix(db): close db conn before reinitializing 2023-12-11 16:14:25 +11:00
e461f9925e feat(db): invert backup/restore logic
Do the migration on a temp copy of the db, then back up the original and move the temp into its file.
2023-12-11 16:14:25 +11:00
abeb1bd3b3 feat(db): reduce power MigrateCallback, only gets cursor
use partial to provide extra dependencies for the image workflow migration function
2023-12-11 16:14:25 +11:00
83e820d721 feat(db): decouple from SqliteDatabase 2023-12-11 16:14:25 +11:00
f8e4b93a74 feat(db): add migration lock file 2023-12-11 16:14:25 +11:00
0710ec30cf feat(db): incorporate feedback 2023-12-11 16:14:25 +11:00
c382329e8c feat(db): move migrator out of SqliteDatabase 2023-12-11 16:14:25 +11:00
f2c6819d68 feat(db): add SQLiteMigrator to perform db migrations 2023-12-11 16:14:25 +11:00