This allows the stats to be written to disk as JSON and analyzed.
- Add dataclasses to hold stats.
- Move stats pretty-print logic to `__str__` of the new `InvocationStatsSummary` class.
- Add `get_stats` and `dump_stats` methods to `InvocationStatsServiceBase`.
- `InvocationStatsService` now throws if stats are requested for a session it doesn't know about. This avoids needing to do a lot of messy null checks.
- Update `DefaultInvocationProcessor` to use the new stats methods and suppresses the new errors.
* redo top panel of workflow editor
* add checkbox option to save to project, integrate save-as flow into first time saving workflow
* remove log
* remove workflowLibrary as a feature that can be disabled
* lint
* feat(ui): make SaveWorkflowAsDialog a singleton
Fixes an issue where the workflow name would erroneously be an empty string (which it should show the current workflow name).
Also makes it easier to interact with this component.
- Extract the dialog state to a hook
- Render the dialog once in `<NodeEditor />`
- Use the hook in the various buttons that should open the dialog
- Fix a few wonkily named components (pre-existing issue)
* fix(ui): when saving a never-before-saved workflow, do not append " (copy)" to the name
* fix(ui): do not obscure workflow library button with add node popover
This component is kinda janky :/ the popover content somehow renders invisibly over the button. I think it's related to the `<PopoverAnchor />.
Need to redo this in the future, but for now, making the popover render lazily fixes this.
---------
Co-authored-by: Mary Hipp <maryhipp@Marys-MacBook-Air.local>
Co-authored-by: psychedelicious <4822129+psychedelicious@users.noreply.github.com>
Adds adds ctrl/meta + scroll to change brush size on canvas.
* changed hotkeys
* new hotkey as an additional
* lint fixed"
* added ctrl scroll and removed hotkey
* using
* added fix
* feedbck_changes
* brush size change logic
* feat(ui): also check for meta key when modifying brush size
* feat(ui): add comment linking to where brush size algo was determined
---------
Co-authored-by: psychedelicious <4822129+psychedelicious@users.noreply.github.com>
Currently translated at 10.5% (151 of 1426 strings)
translationBot(ui): update translation (Turkish)
Currently translated at 8.1% (116 of 1426 strings)
translationBot(ui): update translation (Turkish)
Currently translated at 6.6% (95 of 1426 strings)
Co-authored-by: Ufuk Sarp Selçok <ilkel@live.com>
Translate-URL: https://hosted.weblate.org/projects/invokeai/web-ui/tr/
Translation: InvokeAI/Web UI
It doesn't work now that the theme is external. I'm not sure how to fix it and not sure if it really did much (I don't think I ever got autocomplete...). Maybe it can be implemented in `@invoke-ai/ui-library`.
* dont show duplicate toasts if workflow actions fail due to auth
* dynamic order by options based on projectId
* add endpointName to authtoast to makeit unique per endpoint
* lint
* update toast logic to check based on endpoint name w type safety
* fix save as endpoit name
* lint
* fix type
---------
Co-authored-by: Mary Hipp <maryhipp@Marys-MacBook-Air.local>
* retain id through workflow state so that we correctly update or save new
* lint
---------
Co-authored-by: Mary Hipp <maryhipp@Marys-MacBook-Air.local>
* feat: ✨ "Remix Image" option on images
Adds a new "remix image" option where applicable, recalls all metadata except the seed
* refactor: 🚨 lint code
* feat: ✨ "Remix Image" option on images
Adds a new "remix image" option where applicable, recalls all metadata except the seed
* refactor: 🚨 lint code
* feat: ✨ add new remix hotkey to hotkeys modal
---------
Co-authored-by: psychedelicious <4822129+psychedelicious@users.noreply.github.com>
Currently translated at 60.0% (850 of 1415 strings)
Co-authored-by: Alexander Eichhorn <pfannkuchensack@einfach-doof.de>
Translate-URL: https://hosted.weblate.org/projects/invokeai/web-ui/de/
Translation: InvokeAI/Web UI
Remove `trim()` from model identifier schema, which prevented parsed model identifiers from matching.
The root issue here is that model names are identifiers. This will be resolved in the model manager refactor.
Closes#5556
- Bump `@invoke-ai/ui` for updated styles
- Update regex to parse prompts with newlines
- Update styling of overlay button when prompt has an error
- Fix bug where loading and error state sometimes weren't cleared
We had a one-behind issue with recalling metadata items that had a model.
For example, when recalling LoRAs, we check against the current main model to decide whether or not the requested LoRA is compatible and may be recalled.
When recalling all params, we are often also recalling the main model, but the compat logic didn't compare against this new main model.
The logic is updated to check against the new main model, if one is being set.
Closes#5512
The Ideal Size node is useful for High-Res Optimization as it gives the optimum size for creating an initial generation with minimal artifacts (duplication and other strangeness) from today's models.
After inclusion, front end graph generation can be simplified by offloading calculations for HRO initial generation to this node.
The previous method wasn't totally foolproof, and locales/assets were cached.
To solve this once and for all (famous last words, I know), we can subclass `StaticFiles` and use maximally strict no-caching headers to disable caching on all static files.
Currently translated at 97.3% (1365 of 1402 strings)
translationBot(ui): update translation (Italian)
Currently translated at 97.3% (1365 of 1402 strings)
translationBot(ui): update translation (Italian)
Currently translated at 97.3% (1365 of 1402 strings)
translationBot(ui): update translation (Italian)
Currently translated at 97.3% (1365 of 1402 strings)
Co-authored-by: Riccardo Giovanetti <riccardo.giovanetti@gmail.com>
Translate-URL: https://hosted.weblate.org/projects/invokeai/web-ui/it/
Translation: InvokeAI/Web UI
* resolved conflicts
* changed logo and some design changes
* feedback changes
* resolved conflicts
* changed logo and some design changes
* feedback changes
* lint fixed
* added translations
* some requested changes done
* all feedback changes done and replace links in settingsmenu comp
* fixed the gap between deps verisons & chnaged heights
* feat(ui): minor about modal styling
* feat(ui): tag app endpoints with FetchOnReconnect
* fix(ui): remove unused translation string
---------
Co-authored-by: psychedelicious <4822129+psychedelicious@users.noreply.github.com>
Co-authored-by: Kent Keirsey <31807370+hipsterusername@users.noreply.github.com>
Use each language's own language for their option in the language select. This falls back to the english translation if the language name isn't translated.
* add basic functionality for model metadata fetching from hf and civitai
* add storage
* start unit tests
* add unit tests and documentation
* add missing dependency for pytests
* remove redundant fetch; add modified/published dates; updated docs
* add code to select diffusers files based on the variant type
* implement Civitai installs
* make huggingface parallel downloading work
* add unit tests for model installation manager
- Fixed race condition on selection of download destination path
- Add fixtures common to several model_manager_2 unit tests
- Added dummy model files for testing diffusers and safetensors downloading/probing
- Refactored code for selecting proper variant from list of huggingface repo files
- Regrouped ordering of methods in model_install_default.py
* improve Civitai model downloading
- Provide a better error message when Civitai requires an access token (doesn't give a 403 forbidden, but redirects
to the HTML of an authorization page -- arrgh)
- Handle case of Civitai providing a primary download link plus additional links for VAEs, config files, etc
* add routes for retrieving metadata and tags
* code tidying and documentation
* fix ruff errors
* add file needed to maintain test root diretory in repo for unit tests
* fix self->cls in classmethod
* add pydantic plugin for mypy
* use TestSession instead of requests.Session to prevent any internet activity
improve logging
fix error message formatting
fix logging again
fix forward vs reverse slash issue in Windows install tests
* Several fixes of problems detected during PR review:
- Implement cancel_model_install_job and get_model_install_job routes
to allow for better control of model download and install.
- Fix thread deadlock that occurred after cancelling an install.
- Remove unneeded pytest_plugins section from tests/conftest.py
- Remove unused _in_terminal_state() from model_install_default.
- Remove outdated documentation from several spots.
- Add workaround for Civitai API results which don't return correct
URL for the default model.
* fix docs and tests to match get_job_by_source() rather than get_job()
* Update invokeai/backend/model_manager/metadata/fetch/huggingface.py
Co-authored-by: Ryan Dick <ryanjdick3@gmail.com>
* Call CivitaiMetadata.model_validate_json() directly
Co-authored-by: Ryan Dick <ryanjdick3@gmail.com>
* Second round of revisions suggested by @ryanjdick:
- Fix type mismatch in `list_all_metadata()` route.
- Do not have a default value for the model install job id
- Remove static class variable declarations from non Pydantic classes
- Change `id` field to `model_id` for the sqlite3 `model_tags` table.
- Changed AFTER DELETE triggers to ON DELETE CASCADE for the metadata and tags tables.
- Made the `id` field of the `model_metadata` table into a primary key to achieve uniqueness.
* Code cleanup suggested in PR review:
- Narrowed the declaration of the `parts` attribute of the download progress event
- Removed auto-conversion of str to Url in Url-containing sources
- Fixed handling of `InvalidModelConfigException`
- Made unknown sources raise `NotImplementedError` rather than `Exception`
- Improved status reporting on cached HuggingFace access tokens
* Multiple fixes:
- `job.total_size` returns a valid size for locally installed models
- new route `list_models` returns a paged summary of model, name,
description, tags and other essential info
- fix a few type errors
* consolidated all invokeai root pytest fixtures into a single location
* Update invokeai/backend/model_manager/metadata/metadata_store.py
Co-authored-by: psychedelicious <4822129+psychedelicious@users.noreply.github.com>
* Small tweaks in response to review comments:
- Remove flake8 configuration from pyproject.toml
- Use `id` rather than `modelId` for huggingface `ModelInfo` object
- Use `last_modified` rather than `LastModified` for huggingface `ModelInfo` object
- Add `sha256` field to file metadata downloaded from huggingface
- Add `Invoker` argument to the model installer `start()` and `stop()` routines
(but made it optional in order to facilitate use of the service outside the API)
- Removed redundant `PRAGMA foreign_keys` from metadata store initialization code.
* Additional tweaks and minor bug fixes
- Fix calculation of aggregate diffusers model size to only count the
size of files, not files + directories (which gives different unit test
results on different filesystems).
- Refactor _get_metadata() and _get_download_urls() to have distinct code paths
for Civitai, HuggingFace and URL sources.
- Forward the `inplace` flag from the source to the job and added unit test for this.
- Attach cached model metadata to the job rather than to the model install service.
* fix unit test that was breaking on windows due to CR/LF changing size of test json files
* fix ruff formatting
* a few last minor fixes before merging:
- Turn job `error` and `error_type` into properties derived from the exception.
- Add TODO comment about the reason for handling temporary directory destruction
manually rather than using tempfile.tmpdir().
* add unit tests for reporting HTTP download errors
---------
Co-authored-by: Lincoln Stein <lstein@gmail.com>
Co-authored-by: Ryan Dick <ryanjdick3@gmail.com>
Co-authored-by: psychedelicious <4822129+psychedelicious@users.noreply.github.com>
Per user feedback, this is preferrable to letting them expand when the window grows.
Also bumps `react-resizable-panels` now that one of my PRs is merged to fix an issue.
* feat: allow bfloat16 to be configurable in invoke.yaml
* fix: `torch_dtype()` util
- Use `choose_precision` to get the precision string
- Do not reference deprecated `config.full_precision` flat (why does this still exist?), if a user had this enabled it would override their actual precision setting and potentially cause a lot of confusion.
---------
Co-authored-by: psychedelicious <4822129+psychedelicious@users.noreply.github.com>
- Add various brand images, organise images
- Create favicon for docs pages (light blue version of key logo)
- Rename app title to `Invoke - Community Edition`
Add `FetchOnReconnect` tag, tagging relevant queries with it. This tag is invalidated in the socketConnected listener, when it is determined that the queue changed.
- Add checks to the "recovery" logic for socket connect events to reduce the number of network requests.
- Remove the `isInitialized` state from `systemSlice` and make it a nanostore local to the socketConnected listener. It didn't need to be global state. It's also now more clearly named `isFirstConnection`.
- Export the queue status selector (minor improvement, memoizes it correctly).
- Fixed a bug where after you load more, changing boards doesn't work. The offset and limit for the list image query had some wonky logic, now resolved.
- Addressed major lag in gallery when selecting an image.
Both issues were related to the useMultiselect and useGalleryImages hooks, which caused every image in the gallery to re-render on whenever the selection changed. There's no way to memoize away this - we need to know when the selection changes. This is a longstanding issue.
The selection is only used in a callback, though - the onClick handler for an image to select it (or add it to the existing selection). We don't really need the reactivity for a callback, so we don't need to listen for changes to the selection.
The logic to handle multiple selection is moved to a new `galleryImageClicked` listener, which does all the selection right when it is needed.
The result is that gallery images no long need to do heavy re-renders on any selection change.
Besides the multiselect click handler, there was also inefficient use of DND payloads. Previously, the `IMAGE_DTOS` type had a payload of image DTO objects. This was only used to drag gallery selection into a board. There is no need to hold onto image DTOs when we have the selection state already in redux. We were recalculating this payload for every image, on every tick.
This payload is now just the board id (the only piece of information we need for this particular DND event).
- I also removed some unused DND types while making this change.
There was a lot of convoluted, janky logic related to trying to not mount the context menu's portal until its needed. This was in the library where the component was originally copied from.
I've removed that and resolved the jank, at the cost of there being an extra portal for each instance of the context menu. Don't think this is going to be an issue. If it is, the whole context menu could be refactored to be a singleton.
* ci: add docker build timout; log free space on runner before and after build
* docker: bump frontend builder to node=20.x; skip linting on build
* chore: gitignore .pnpm-store
* update code owners for docker and CI
---------
Co-authored-by: Millun Atluri <Millu@users.noreply.github.com>
I was troubleshooting a hotkeys issue on canvas and thought I had broken the tool logic in a past change so I redid it moving it to nanostores. In the end, the issue was an upstream but with the hotkeys library, but I like having tool in nanostores so I'm leaving it.
It's ephemeral interaction state anyways, doesn't need to be in redux.
There's a challenge to accomplish this due to our slice structure - the model is stored in `generationSlice`, but `canvasSlice` also needs to have awareness of it. For example, when the model changes, the canvas slice doesn't know what the previous model was, so it doesn't know whether or not to optimize the size.
This means we need to lift the "should we optimize size" information up. To do this, the `modelChanged` action creator accepts the previous model as an optional second arg.
Now the canvas has access to both the previous model and new model selection, and can decide whether or not it should optimize its size setting in the same way that the generation slice does.
Closes #5452
For some reason `ReturnType<typeof useListImagesQuery>` isn't working correctly, and destructuring `queryResult` it results in `any`, when the hook is used.
I've removed the explicit return typing so that consumers of the hook get correct types.
Workflow building would fail when a current image node was in the workflow due to the strict validation.
So we need to use the other workflow builder util first, which strips out extraneous data.
This bug was introduced during an attempt to optimize the workflow building logic, which was causing slowdowns on the workflow editor.
* do not show toast if 403 is triggered by lack of image access
* remove log
* lint
---------
Co-authored-by: Mary Hipp <maryhipp@Marys-MacBook-Air.local>
* feat(ui): get rid of convoluted socket vs appSocket redux actions
There's no need to have `socket...` and `appSocket...` actions.
I did this initially due to a misunderstanding about the sequence of handling from middleware to reducers.
* feat(ui): bump deps
Mainly bumping to get latest `redux-remember`.
A change to socket.io required a change to the types in `useSocketIO`.
* chore(ui): format
* feat(ui): add error handling to redux persistence layer
- Add an error handler to `redux-remember` config using our logger
- Add custom errors representing storage set and get failures
- Update storage driver to raise these accordingly
- wrap method to clear idbkeyval storage and tidy its logic up
* feat(ui): add debuggingLoggerMiddleware
This simply logs every action and a diff of the state change.
Due to the noise this creates, it's not added by default at all. Add it to the middlewares if you want to use it.
* feat(ui): add $socket to window if in dev mode
* fix(ui): do not enable cancel hotkeys on inputs
* fix(ui): use JSON.stringify for ROARR logger serializer
A recent change to ROARR introduced limits to the size of data that will logged. This ends up making our logs far less useful. Change the serializer back to what it was previously.
* feat(ui): change diff util, update debuggerLoggerMiddleware
The previous diff library would present deleted things as `undefined`. Unfortunately, a JSON.stringify cycle will strip those values out. The ROARR logger does this and so the diffs end up being a lot less useful, not showing removed keys.
The new diff library uses a different format for the delta that serializes nicely.
* feat(ui): add migrations to redux persistence layer
- All persisted slices must now have a slice config, consisting of their initial state and a migrate callback. The migrate callback is very simple for now, with no type safety. It adds missing properties to the state. A future enhancement might be to model the each slice's state with e.g. zod and have proper validation and types.
- Persisted slices now have a `_version` property
- The migrate callback is called inside `redux-remember`'s `unserialize` handler. I couldn't figure out a good way to put this into the reducer and do logging (reducers should have no side effects). Also I ran into a weird race condition that I couldn't figure out. And finally, the typings are tricky. This works for now.
- `generationSlice` and `canvasSlice` both need migrations for the new aspect ratio setup, this has been added
- Stuff related to persistence has been moved in to `store.ts` for simplicity
* feat(ui): clean up StorageError class
* fix(ui): scale method default is now 'auto'
* feat(ui): when changing controlnet model, enable autoconfig
* fix(ui): make embedding popover immediately accessible
Prevents hotkeys from being captured when embeddings are still loading.
