Commit Graph

481 Commits

Author SHA1 Message Date
psychedelicious
4082f25062 feat(ui): do not optimize size when changing between models with same base model
There's a challenge to accomplish this due to our slice structure - the model is stored in `generationSlice`, but `canvasSlice` also needs to have awareness of it. For example, when the model changes, the canvas slice doesn't know what the previous model was, so it doesn't know whether or not to optimize the size.

This means we need to lift the "should we optimize size" information up. To do this, the `modelChanged` action creator accepts the previous model as an optional second arg.

Now the canvas has access to both the previous model and new model selection, and can decide whether or not it should optimize its size setting in the same way that the generation slice does.

Closes  #5452
2024-01-10 00:11:05 -05:00
psychedelicious
63d74b4ba6 feat(ui): remove unnecessary tabChanged listener
This was needed when we didn't support SDXL on canvas.
2024-01-10 00:11:05 -05:00
psychedelicious
0809e832d4 fix(ui): use less brutally strict workflow validation
Workflow building would fail when a current image node was in the workflow due to the strict validation.

So we need to use the other workflow builder util first, which strips out extraneous data.

This bug was introduced during an attempt to optimize the workflow building logic, which was causing slowdowns on the workflow editor.
2024-01-10 14:31:14 +11:00
psychedelicious
0fc08bb384
ui: redesign followups 8 (#5445)
* feat(ui): get rid of convoluted socket vs appSocket redux actions

There's no need to have `socket...` and `appSocket...` actions.

I did this initially due to a misunderstanding about the sequence of handling from middleware to reducers.

* feat(ui): bump deps

Mainly bumping to get latest `redux-remember`.

A change to socket.io required a change to the types in `useSocketIO`.

* chore(ui): format

* feat(ui): add error handling to redux persistence layer

- Add an error handler to `redux-remember` config using our logger
- Add custom errors representing storage set and get failures
- Update storage driver to raise these accordingly
- wrap method to clear idbkeyval storage and tidy its logic up

* feat(ui): add debuggingLoggerMiddleware

This simply logs every action and a diff of the state change.

Due to the noise this creates, it's not added by default at all. Add it to the middlewares if you want to use it.

* feat(ui): add $socket to window if in dev mode

* fix(ui): do not enable cancel hotkeys on inputs

* fix(ui): use JSON.stringify for ROARR logger serializer

A recent change to ROARR introduced limits to the size of data that will logged. This ends up making our logs far less useful. Change the serializer back to what it was previously.

* feat(ui): change diff util, update debuggerLoggerMiddleware

The previous diff library would present deleted things as `undefined`. Unfortunately, a JSON.stringify cycle will strip those values out. The ROARR logger does this and so the diffs end up being a lot less useful, not showing removed keys.

The new diff library uses a different format for the delta that serializes nicely.

* feat(ui): add migrations to redux persistence layer

- All persisted slices must now have a slice config, consisting of their initial state and a migrate callback. The migrate callback is very simple for now, with no type safety. It adds missing properties to the state. A future enhancement might be to model the each slice's state with e.g. zod and have proper validation and types.
- Persisted slices now have a `_version` property
- The migrate callback is called inside `redux-remember`'s `unserialize` handler. I couldn't figure out a good way to put this into the reducer and do logging (reducers should have no side effects). Also I ran into a weird race condition that I couldn't figure out. And finally, the typings are tricky. This works for now.
- `generationSlice` and `canvasSlice` both need migrations for the new aspect ratio setup, this has been added
- Stuff related to persistence has been moved in to `store.ts` for simplicity

* feat(ui): clean up StorageError class

* fix(ui): scale method default is now 'auto'

* feat(ui): when changing controlnet model, enable autoconfig

* fix(ui): make embedding popover immediately accessible

Prevents hotkeys from being captured when embeddings are still loading.
2024-01-08 09:11:45 -05:00
psychedelicious
3428ea1b3c feat(ui): use config for all numerical params
Centralize the initial/min/max/etc values for all numerical params. We used this for some but at some point stopped updating it.

All numerical params now use their respective configs. Far fewer hardcoded values throughout the app now.

Also updated the config types a bit to better accommodate slider vs number input constraints.
2024-01-07 13:49:29 +11:00
psychedelicious
2c049a3b94 feat(ui): clean up a few selectors that do not need to be memoized 2024-01-06 00:03:07 +11:00
psychedelicious
367de44a8b fix(ui): tidy remaining selectors
These were just using overly verbose syntax - like explicitly typing `state: RootState`, which is unnecessary.
2024-01-06 00:03:07 +11:00
psychedelicious
f5f378d04b fix(ui): revert back to lrumemoize 2024-01-06 00:03:07 +11:00
psychedelicious
a23502f7ff fix(ui): do not use state => state as an input selector
This is a no-no, whoops!
2024-01-06 00:03:07 +11:00
psychedelicious
ce64dbefce chore(ui): lint 2024-01-06 00:03:07 +11:00
psychedelicious
6924b04d7c feat(ui): use lruMemoize for all entity adapter selectors 2024-01-06 00:03:07 +11:00
Mary Hipp
5d475a40f5 lint 2024-01-04 12:51:36 -05:00
Mary Hipp
bca7ea1674 option to override logo component 2024-01-04 12:51:36 -05:00
Mary Hipp Rogers
9e2e740033
custom components for nav, gallery header, and app info (#5400)
* replace custom header with custom nav component to go below settings

* add option for custom gallery header

* add option for custom app info text on logo hover

* add data-testid for tabs

* remove descriptions

* lint

* lint

---------

Co-authored-by: Mary Hipp <maryhipp@Marys-MacBook-Air.local>
2024-01-04 15:30:27 +00:00
psychedelicious
8cf14287b6 feat(ui): simplify App.tsx layout
There was an extra div, needed for the fullscreen file upload dropzone, that made styling the main app containers a bit awkward.

Refactor the uploader a bit to simplify this - no longer need so many app-level wrappers. Much cleaner.
2024-01-04 09:36:59 -05:00
blessedcoolant
2250bca8d9 feat: Remove Header
Remove header and incorporate everything else into the side bar and other areas
2024-01-04 09:36:59 -05:00
psychedelicious
644e9287f0 chore(ui): control adapter docstrings 2024-01-04 13:59:25 +11:00
psychedelicious
77c5b051f0 fix(ui): clean up actionsDenylist 2024-01-03 09:09:50 -05:00
psychedelicious
4fdc4c15f9 feat(ui): add optimal size handling 2024-01-03 09:09:50 -05:00
psychedelicious
46d830b9fa feat(ui): new logo! 2024-01-03 13:18:50 +11:00
psychedelicious
ebe717099e feat(ui): add $store to window in dev mode
Helpful for troubleshooting.
2024-01-03 13:18:50 +11:00
psychedelicious
4a43e1c1b8 fix(ui): restore global hotkeys 2024-01-03 13:18:50 +11:00
Mary Hipp
a408f562d6 option to use new brand for loader 2024-01-02 10:02:33 -05:00
psychedelicious
870cc5b733 feat(ui): dynamic prompts loading ux
- Prompt must have an open curly brace followed by a close curly brace to enable dynamic prompts processing
- If a the given prompt already had a dynamic prompt cached, do not re-process
- If processing is not needed, user may invoke immediately
- Invoke button shows loading state when dynamic prompts are processing, tooltip says generating
- Dynamic prompts preview icon in prompt box shows loading state when processing, tooltip says generating
2024-01-02 07:28:53 -05:00
psychedelicious
e9bce6e1c3 fix(ui): fix cut off badge on queue actions menu 2024-01-02 07:28:53 -05:00
psychedelicious
9169006171 chore(ui): lint 2024-01-01 08:13:23 -05:00
psychedelicious
9359c03c3c feat(ui): use zod-less workflow builder when appropriate 2024-01-01 08:13:23 -05:00
psychedelicious
e698a8006c feat(ui): use lruMemoize for argsMemoize on selectors
This provides a small performance improvement, on the order of a few ms per interaction.
2024-01-01 08:13:23 -05:00
psychedelicious
5d4610d981 feat(ui): store node templates in separate slice
Flattens the `nodes` slice. May offer minor perf improvements in addition to just being cleaner.
2024-01-01 08:13:23 -05:00
psychedelicious
bd92a31d15 feat(ui): add createLruSelector
This uses the previous implementation of the memoization function in reselect. It's possible for the new weakmap-based memoization to cause memory leaks in certain scenarios, so we will avoid it for now.
2024-01-01 08:13:23 -05:00
psychedelicious
4f2930412e feat(ui): use primitive style props or memoized sx objects 2023-12-29 08:26:14 -05:00
psychedelicious
83049a3a5b fix(ui): typo in canvas model handler 2023-12-29 08:26:14 -05:00
psychedelicious
f0b102d830 feat(ui): ux improvements & redesign
This is a squash merge of a bajillion messy small commits created while iterating on the UI component library and redesign.
2023-12-29 08:26:14 -05:00
Mary Hipp
562fb1f3a1 add authToastMiddleware back and fix parsing 2023-12-20 14:59:33 -05:00
psychedelicious
1af4260ab6 fix(ui): fix workflow saving
'id' should not be omitted when building a workflow, it makes workflows always save as a copy
2023-12-09 16:35:44 +11:00
psychedelicious
72cb8b83fe feat(ui): upgrade redux and RTK
There are a few breaking changes, which I've addressed.

The vast majority of changes are related to new handling of `reselect`'s `createSelector` options.

For better or worse, we memoize just about all our selectors using lodash `isEqual` for `resultEqualityCheck`. The upgrade requires we explicitly set the `memoize` option to `lruMemoize` to continue using lodash here.

Doing that required changing our `defaultSelectorOptions`.

Instead of changing that and finding dozens of instances where we weren't using that and instead were defining selector options manually, I've created a pre-configured selector: `createMemoizedSelector`.

This is now used everywhere instead of `createSelector`.
2023-12-09 16:09:26 +11:00
psychedelicious
95a3c89a56 chore(ui): lint 2023-12-09 16:03:09 +11:00
psychedelicious
5902a52e40 feat(ui): add storybook 2023-12-09 16:03:09 +11:00
psychedelicious
c42d692ea6
feat: workflow library (#5148)
* chore: bump pydantic to 2.5.2

This release fixes pydantic/pydantic#8175 and allows us to use `JsonValue`

* fix(ui): exclude public/en.json from prettier config

* fix(workflow_records): fix SQLite workflow insertion to ignore duplicates

* feat(backend): update workflows handling

Update workflows handling for Workflow Library.

**Updated Workflow Storage**

"Embedded Workflows" are workflows associated with images, and are now only stored in the image files. "Library Workflows" are not associated with images, and are stored only in DB.

This works out nicely. We have always saved workflows to files, but recently began saving them to the DB in addition to in image files. When that happened, we stopped reading workflows from files, so all the workflows that only existed in images were inaccessible. With this change, access to those workflows is restored, and no workflows are lost.

**Updated Workflow Handling in Nodes**

Prior to this change, workflows were embedded in images by passing the whole workflow JSON to a special workflow field on a node. In the node's `invoke()` function, the node was able to access this workflow and save it with the image. This (inaccurately) models workflows as a property of an image and is rather awkward technically.

A workflow is now a property of a batch/session queue item. It is available in the InvocationContext and therefore available to all nodes during `invoke()`.

**Database Migrations**

Added a `SQLiteMigrator` class to handle database migrations. Migrations were needed to accomodate the DB-related changes in this PR. See the code for details.

The `images`, `workflows` and `session_queue` tables required migrations for this PR, and are using the new migrator. Other tables/services are still creating tables themselves. A followup PR will adapt them to use the migrator.

**Other/Support Changes**

- Add a `has_workflow` column to `images` table to indicate that the image has an embedded workflow.
- Add handling for retrieving the workflow from an image in python. The image file must be fetched, the workflow extracted, and then sent to client, avoiding needing the browser to parse the image file. With the `has_workflow` column, the UI knows if there is a workflow to be fetched, and only fetches when the user requests to load the workflow.
- Add route to get the workflow from an image
- Add CRUD service/routes for the library workflows
- `workflow_images` table and services removed (no longer needed now that embedded workflows are not in the DB)

* feat(ui): updated workflow handling (WIP)

Clientside updates for the backend workflow changes.

Includes roughed-out workflow library UI.

* feat: revert SQLiteMigrator class

Will pursue this in a separate PR.

* feat(nodes): do not overwrite custom node module names

Use a different, simpler method to detect if a node is custom.

* feat(nodes): restore WithWorkflow as no-op class

This class is deprecated and no longer needed. Set its workflow attr value to None (meaning it is now a no-op), and issue a warning when an invocation subclasses it.

* fix(nodes): fix get_workflow from queue item dict func

* feat(backend): add WorkflowRecordListItemDTO

This is the id, name, description, created at and updated at workflow columns/attrs. Used to display lists of workflowsl

* chore(ui): typegen

* feat(ui): add workflow loading, deleting to workflow library UI

* feat(ui): workflow library pagination button styles

* wip

* feat: workflow library WIP

- Save to library
- Duplicate
- Filter/sort
- UI/queries

* feat: workflow library - system graphs - wip

* feat(backend): sync system workflows to db

* fix: merge conflicts

* feat: simplify default workflows

- Rename "system" -> "default"
- Simplify syncing logic
- Update UI to match

* feat(workflows): update default workflows

- Update TextToImage_SD15
- Add TextToImage_SDXL
- Add README

* feat(ui): refine workflow list UI

* fix(workflow_records): typo

* fix(tests): fix tests

* feat(ui): clean up workflow library hooks

* fix(db): fix mis-ordered db cleanup step

It was happening before pruning queue items - should happen afterwards, else you have to restart the app again to free disk space made available by the pruning.

* feat(ui): tweak reset workflow editor translations

* feat(ui): split out workflow redux state

The `nodes` slice is a rather complicated slice. Removing `workflow` makes it a bit more reasonable.

Also helps to flatten state out a bit.

* docs: update default workflows README

* fix: tidy up unused files, unrelated changes

* fix(backend): revert unrelated service organisational changes

* feat(backend): workflow_records.get_many arg "filter_text" -> "query"

* feat(ui): use custom hook in current image buttons

Already in use elsewhere, forgot to use it here.

* fix(ui): remove commented out property

* fix(ui): fix workflow loading

- Different handling for loading from library vs external
- Fix bug where only nodes and edges loaded

* fix(ui): fix save/save-as workflow naming

* fix(ui): fix circular dependency

* fix(db): fix bug with releasing without lock in db.clean()

* fix(db): remove extraneous lock

* chore: bump ruff

* fix(workflow_records): default `category` to `WorkflowCategory.User`

This allows old workflows to validate when reading them from the db or image files.

* hide workflow library buttons if feature is disabled

---------

Co-authored-by: Mary Hipp <maryhipp@Marys-MacBook-Air.local>
2023-12-09 09:48:38 +11:00
Mary Hipp Rogers
b519b6e1e0
add middleware to handle 403 errors (#5245)
* add middleware to handle 403 errors

* remove log

* add logic to warn the user if not all requested images could be deleted

* lint

* fix copy

* feat(ui): simplify batchEnqueuedListener error toast logic

* feat(ui): use translations for error messages

* chore(ui): lint

---------

Co-authored-by: Mary Hipp <maryhipp@Marys-MacBook-Air.local>
Co-authored-by: psychedelicious <4822129+psychedelicious@users.noreply.github.com>
2023-12-07 19:26:15 -05:00
Mary Hipp Rogers
0463541d99
dont set socketURL until socket is initialized (#5229)
* dont set socketURL until socket is initialized

* cleanup

* feat(ui): simplify `socketUrl` memo

no need to mutate the string; just return early if using baseUrl

---------

Co-authored-by: Mary Hipp <maryhipp@Marys-MacBook-Air.local>
Co-authored-by: psychedelicious <4822129+psychedelicious@users.noreply.github.com>
2023-12-04 21:01:49 +00:00
psychedelicious
a8ef4e5be8 fix(ui): fix types and storage prefix 2023-12-01 09:11:48 +11:00
Mary Hipp
e6fe2540b8 dynamically create indexedDB store using unique store key if available 2023-12-01 09:11:48 +11:00
psychedelicious
aadcde3edd feat(ui): use IndexedDB for persistence
IndexedDB has a much larger storage limit than LocalStorage, and is widely supported.

Implemented as a custom storage driver for `redux-remember` via `idb-keyval`. `idb-keyval` is a simple wrapper for IndexedDB that allows it to be used easily as a key-value store.

The logic to clear persisted storage has been updated throughout the app.
2023-12-01 09:11:48 +11:00
psychedelicious
59d932e9c1 chore(ui): lint 2023-11-29 11:06:07 +11:00
psychedelicious
4af4486dd9 feat(nodes,ui): add detection of custom nodes
Custom nodes have a new attribute `node_pack` indicating the node pack they came from.

- This is displayed in the UI in the icon icon tooltip.
- If a workflow is loaded and a node is unavailable, its node pack will be displayed (if it is known).
- If a workflow is migrated from v1 to v2, and the node is unknown, it falls back to "Unknown". If the missing node pack is installed and the node is updated, the node pack will be updated as expected.
2023-11-29 10:49:31 +11:00
psychedelicious
8d99113bef feat(ui): organize node utils 2023-11-29 10:49:31 +11:00
psychedelicious
86a74e929a feat(ui): add support for custom field types
Node authors may now create their own arbitrary/custom field types. Any pydantic model is supported.

Two notes:
1. Your field type's class name must be unique.

Suggest prefixing fields with something related to the node pack as a kind of namespace.

2. Custom field types function as connection-only fields.

For example, if your custom field has string attributes, you will not get a text input for that attribute when you give a node a field with your custom type.

This is the same behaviour as other complex fields that don't have custom UIs in the workflow editor - like, say, a string collection.

feat(ui): fix tooltips for custom types

We need to hold onto the original type of the field so they don't all just show up as "Unknown".

fix(ui): fix ts error with custom fields

feat(ui): custom field types connection validation

In the initial commit, a custom field's original type was added to the *field templates* only as `originalType`. Custom fields' `type` property was `"Custom"`*. This allowed for type safety throughout the UI logic.

*Actually, it was `"Unknown"`, but I changed it to custom for clarity.

Connection validation logic, however, uses the *field instance* of the node/field. Like the templates, *field instances* with custom types have their `type` set to `"Custom"`, but they didn't have an `originalType` property. As a result, all custom fields could be connected to all other custom fields.

To resolve this, we need to add `originalType` to the *field instances*, then switch the validation logic to use this instead of `type`.

This ended up needing a bit of fanagling:

- If we make `originalType` a required property on field instances, existing workflows will break during connection validation, because they won't have this property. We'd need a new layer of logic to migrate the workflows, adding the new `originalType` property.

While this layer is probably needed anyways, typing `originalType` as optional is much simpler. Workflow migration logic can come layer.

(Technically, we could remove all references to field types from the workflow files, and let the templates hold all this information. This feels like a significant change and I'm reluctant to do it now.)

- Because `originalType` is optional, anywhere we care about the type of a field, we need to use it over `type`. So there are a number of `field.originalType ?? field.type` expressions. This is a bit of a gotcha, we'll need to remember this in the future.

- We use `Array.prototype.includes()` often in the workflow editor, e.g. `COLLECTION_TYPES.includes(type)`. In these cases, the const array is of type `FieldType[]`, and `type` is is `FieldType`.

Because we now support custom types, the arg `type` is now widened from `FieldType` to `string`.

This causes a TS error. This behaviour is somewhat controversial (see https://github.com/microsoft/TypeScript/issues/14520). These expressions are now rewritten as `COLLECTION_TYPES.some((t) => t === type)` to satisfy TS. It's logically equivalent.

fix(ui): typo

feat(ui): add CustomCollection and CustomPolymorphic field types

feat(ui): add validation for CustomCollection & CustomPolymorphic types

- Update connection validation for custom types
- Use simple string parsing to determine if a field is a collection or polymorphic type.
- No longer need to keep a list of collection and polymorphic types.
- Added runtime checks in `baseinvocation.py` to ensure no fields are named in such a way that it could mess up the new parsing

chore(ui): remove errant console.log

fix(ui): rename 'nodes.currentConnectionFieldType' -> 'nodes.connectionStartFieldType'

This was confusingly named and kept tripping me up. Renamed to be consistent with the `reactflow` `ConnectionStartParams` type.

fix(ui): fix ts error

feat(nodes): add runtime check for custom field names

"Custom", "CustomCollection" and "CustomPolymorphic" are reserved field names.

chore(ui): add TODO for revising field type names

wip refactor fieldtype structured

wip refactor field types

wip refactor types

wip refactor types

fix node layout

refactor field types

chore: mypy

organisation

organisation

organisation

fix(nodes): fix field orig_required, field_kind and input statuses

feat(nodes): remove broken implementation of default_factory on InputField

Use of this could break connection validation due to the difference in node schemas required fields and invoke() required args.

Removed entirely for now. It wasn't ever actually used by the system, because all graphs always had values provided for fields where default_factory was used.

Also, pydantic is smart enough to not reuse the same object when specifying a default value - it clones the object first. So, the common pattern of `default_factory=list` is extraneous. It can just be `default=[]`.

fix(nodes): fix InputField name validation

workflow validation

validation

chore: ruff

feat(nodes): fix up baseinvocation comments

fix(ui): improve typing & logic of buildFieldInputTemplate

improved error handling in parseFieldType

fix: back compat for deprecated default_factory and UIType

feat(nodes): do not show node packs loaded log if none loaded

chore(ui): typegen
2023-11-29 10:49:31 +11:00
Rohinish
6e6d903f99
eslint added to enforce translations (#5150)
* eslint added and new string added

* strings and translation hook added

* more changes made

* missing translation added

* final errors resolve in progress

* all errors resolved

* fix(ui): fix missing import of `t()`

* fix(ui): use plurals for moving images to board translation

* fix(ui): fix typo in translation key

* fix(ui): do not use translation for "invoke ai"

* chore(ui): lint

---------

Co-authored-by: psychedelicious <4822129+psychedelicious@users.noreply.github.com>
2023-11-25 14:46:19 +11:00
psychedelicious
785d584603 feat(ui): clean up network stuff
- Remove unused dependency on `openapi-fetch`
- Organise network-related nanostores
2023-11-24 19:30:37 -08:00
psychedelicious
da4aab9233 fix(ui): restore dynamic middleware 2023-11-24 19:30:37 -08:00
psychedelicious
591b601fd3 feat(ui): add debug mode & socketOptions 2023-11-24 19:30:37 -08:00
Rohinish
4d8b8a2db8
fix(ui): add missing translations (#5096)
* first string only to test

* more strings changed

* almost half strings added in json file

* more strings added

* more changes

* few strings and t function changed

* resolved

* errors resolved

* chore(ui): fmt en.json

---------

Co-authored-by: psychedelicious <4822129+psychedelicious@users.noreply.github.com>
2023-11-20 06:24:03 +00:00
psychedelicious
91ef24e15c fix(nodes,ui): fix missed/canvas temp images in gallery
Resolves two bugs introduced in #5106:

1. Linear UI images sometimes didn't make it to the gallery.

This was a race condition. The VAE decode nodes were handled by the socketInvocationComplete listener. At that moment, the image was marked as intermediate. Immediately after this node was handled, a LinearUIOutputInvocation, introduced in #5106, was handled by socketInvocationComplete. This node internally sets changed the image to not intermediate.

During the handling of that socketInvocationComplete, RTK Query would sometimes use its cache instead of retrieving the image DTO again. The result is that the UI never got the message that the image was not intermediate, so it wasn't added to the gallery.

This is resolved by refactoring the socketInvocationComplete listener. We now skip the gallery processing for linear UI events, except for the LinearUIOutputInvocation. Images now always make it to the gallery, and network requests to get image DTOs are substantially reduced.

2. Canvas temp images always went into the gallery

The LinearUIOutputInvocation was always setting its image's is_intermediate to false. This included all canvas images and resulted in all canvas temp images going to gallery.

This is resolved by making LinearUIOutputInvocation set is_intermediate based on `self.is_intermediate`. The behaviour now more or less mirroring the behaviour of is_intermediate on other image-outputting nodes, except it doesn't save the image again - only changes it.

One extra minor change - LinearUIOutputInvocation only changes is_intermediate if it differs from the image's current setting. Very minor optimisation.
2023-11-17 07:32:04 +11:00
psychedelicious
4599517c6c feat: add private node for linear UI image outputting
Add a LinearUIOutputInvocation node to be the new terminal node for Linear UI graphs. This node is private and hidden from the Workflow Editor, as it is an implementation detail.

The Linear UI was using the Save Image node for this purpose. It allowed every linear graph to end a single node type, which handled saving metadata and board. This substantially reduced the complexity of the linear graphs.

This caused two related issues:
- Images were saved to disk twice
- Noticeable delay between when an image was decoded and showed up in the UI

To resolve this, the new LinearUIOutputInvocation node will handle adding an image to a board if one is provided.

Metadata is no longer provided in this unified node. Instead, the metadata graph helpers now need to know the node to add metadata to and provide it to the last node that actually outputs an image. This is a `l2i` node for txt2img & img2img graphs, and a different image-outputting node for canvas graphs.

HRF poses another complication, in that it changes the terminal node. To handle this, a new metadata util is added called `setMetadataReceivingNode()`. HRF calls this to change the node that should receive the graph's metadata.

This resolves the duplicate images issue and improves perf without otherwise changing the user experience.
2023-11-16 18:56:59 +11:00
psychedelicious
7fcf475aec feat(ui): add Update All Nodes button 2023-11-16 12:42:25 +11:00
psychedelicious
bb52861896 chore(ui): move MM components & store to features/
Somehow they had ended up in `features/ui/tabs` which isn't right
2023-11-13 16:32:03 +11:00
Kent Keirsey
67f2616d5a
Merge branch 'main' into revert-4923-revert-4914-feat/mix-cnet-t2iadapter 2023-11-06 07:34:51 -08:00
psychedelicious
f04462973b feat(ui): create debounced metadata/workflow query hooks
Also added config options for metadata and workflow debounce times (`metadataFetchDebounce` & `workflowFetchDebounce`).

Falls back to 0 if not provided.

In OSS, because we have no major latency concerns, the debounce is 0. But in other environments, it may be desirable to set this to something like 300ms.
2023-10-20 12:05:13 +11:00
psychedelicious
284a257c25
feat: remove enqueue_graph routes/methods (#4922)
This is totally extraneous - it's almost identical to `enqueue_batch`.
2023-10-17 18:00:40 +00:00
Kent Keirsey
55ad4feb5c Revert "Revert "feat(ui): remove special handling for t2i vs controlnet""
This reverts commit bdf4c4944c.
2023-10-17 11:59:19 -04:00
psychedelicious
bdf4c4944c Revert "feat(ui): remove special handling for t2i vs controlnet"
This reverts commit b146993553.
2023-10-17 11:59:11 -04:00
psychedelicious
b146993553 feat(ui): remove special handling for t2i vs controlnet 2023-10-17 19:42:06 +11:00
psychedelicious
c238a7f18b feat(api): chore: pydantic & fastapi upgrade
Upgrade pydantic and fastapi to latest.

- pydantic~=2.4.2
- fastapi~=103.2
- fastapi-events~=0.9.1

**Big Changes**

There are a number of logic changes needed to support pydantic v2. Most changes are very simple, like using the new methods to serialized and deserialize models, but there are a few more complex changes.

**Invocations**

The biggest change relates to invocation creation, instantiation and validation.

Because pydantic v2 moves all validation logic into the rust pydantic-core, we may no longer directly stick our fingers into the validation pie.

Previously, we (ab)used models and fields to allow invocation fields to be optional at instantiation, but required when `invoke()` is called. We directly manipulated the fields and invocation models when calling `invoke()`.

With pydantic v2, this is much more involved. Changes to the python wrapper do not propagate down to the rust validation logic - you have to rebuild the model. This causes problem with concurrent access to the invocation classes and is not a free operation.

This logic has been totally refactored and we do not need to change the model any more. The details are in `baseinvocation.py`, in the `InputField` function and `BaseInvocation.invoke_internal()` method.

In the end, this implementation is cleaner.

**Invocation Fields**

In pydantic v2, you can no longer directly add or remove fields from a model.

Previously, we did this to add the `type` field to invocations.

**Invocation Decorators**

With pydantic v2, we instead use the imperative `create_model()` API to create a new model with the additional field. This is done in `baseinvocation.py` in the `invocation()` wrapper.

A similar technique is used for `invocation_output()`.

**Minor Changes**

There are a number of minor changes around the pydantic v2 models API.

**Protected `model_` Namespace**

All models' pydantic-provided methods and attributes are prefixed with `model_` and this is considered a protected namespace. This causes some conflict, because "model" means something to us, and we have a ton of pydantic models with attributes starting with "model_".

Forunately, there are no direct conflicts. However, in any pydantic model where we define an attribute or method that starts with "model_", we must tell set the protected namespaces to an empty tuple.

```py
class IPAdapterModelField(BaseModel):
    model_name: str = Field(description="Name of the IP-Adapter model")
    base_model: BaseModelType = Field(description="Base model")

    model_config = ConfigDict(protected_namespaces=())
```

**Model Serialization**

Pydantic models no longer have `Model.dict()` or `Model.json()`.

Instead, we use `Model.model_dump()` or `Model.model_dump_json()`.

**Model Deserialization**

Pydantic models no longer have `Model.parse_obj()` or `Model.parse_raw()`, and there are no `parse_raw_as()` or `parse_obj_as()` functions.

Instead, you need to create a `TypeAdapter` object to parse python objects or JSON into a model.

```py
adapter_graph = TypeAdapter(Graph)
deserialized_graph_from_json = adapter_graph.validate_json(graph_json)
deserialized_graph_from_dict = adapter_graph.validate_python(graph_dict)
```

**Field Customisation**

Pydantic `Field`s no longer accept arbitrary args.

Now, you must put all additional arbitrary args in a `json_schema_extra` arg on the field.

**Schema Customisation**

FastAPI and pydantic schema generation now follows the OpenAPI version 3.1 spec.

This necessitates two changes:
- Our schema customization logic has been revised
- Schema parsing to build node templates has been revised

The specific aren't important, but this does present additional surface area for bugs.

**Performance Improvements**

Pydantic v2 is a full rewrite with a rust backend. This offers a substantial performance improvement (pydantic claims 5x to 50x depending on the task). We'll notice this the most during serialization and deserialization of sessions/graphs, which happens very very often - a couple times per node.

I haven't done any benchmarks, but anecdotally, graph execution is much faster. Also, very larges graphs - like with massive iterators - are much, much faster.
2023-10-17 14:59:25 +11:00
psychedelicious
3e389d3f60 chore(ui): update deps 2023-10-15 19:30:39 +11:00
Paul Curry
75663ec81e
feat (ui, generation): High Resolution Fix MVP in Text2Image Linear Flow (#4819)
* added HrfScale type with initial value

* working

* working

* working

* working

* working

* added addHrfToGraph

* continueing to implement this

* working on this

* comments

* working

* made hrf into its own collapse

* working on adding strength slider

* working

* working

* refactoring

* working

* change of this working: 0

* removed onnx support since apparently its not used

* working

* made scale integer

* trying out psycicpebbles idea

* working

* working on this

* working

* added toggle

* comments

* self review

* fixing things

* remove 'any' type

* fixing typing

* changed initial strength value to 3 (large values cause issues)

* set denoising start to be 1 - strength to resemble image to image

* set initial value

* added image to image

* pr1

* pr2

* updating to resolution finding

* working

* working

* working

* working

* working

* working

* working

* working

* working

* use memo

* connect rescale hw to noise

* working

* fixed min bug

* nit

* hides elements conditionally

* style

* feat(ui): add config for HRF, disable if feature disabled or ONNX model in use

* fix(ui): use `useCallback` for HRF toggle

---------

Co-authored-by: psychedelicious <4822129+psychedelicious@users.noreply.github.com>
2023-10-14 10:34:41 +00:00
psychedelicious
75de20af6a fix(ui): fix plurals in translation 2023-10-12 21:34:24 +11:00
Mary Hipp Rogers
69937d68d2
Maryhipp/dummy bulk download (#4852)
* UI for bulk downloading boards or groups of images

* placeholder route for bulk downloads that does nothing

* lint

---------

Co-authored-by: Mary Hipp <maryhipp@Marys-MacBook-Air.local>
2023-10-11 23:27:22 +00:00
psychedelicious
1f751f8c21 fix(ui): remove extraneous cache update 2023-10-09 20:11:21 +11:00
psychedelicious
ca95a3bd0d fix(ui): fix canvas soft-lock if canceled before first generation
The canvas needs to be set to staging mode as soon as a canvas-destined batch is enqueued. If the batch is is fully canceled before an image is generated, we need to remove that batch from the canvas `batchIds` watchlist, else canvas gets stuck in staging mode with no way to exit.

The changes here allow the batch status to be tracked, and if a batch has all its items completed, we can remove it from the `batchIds` watchlist. The `batchIds` watchlist now accurately represents *incomplete* canvas batches, fixing this cause of soft lock.
2023-10-09 20:11:21 +11:00
psychedelicious
1cc686734b feat(ui): on base model change, disable control adapters
Previously it deleted them entirely.
2023-10-07 22:30:15 +11:00
psychedelicious
82e8b92ba0 feat(ui): display toast when enabling t2i/controlnet and disabling the other 2023-10-07 22:30:15 +11:00
psychedelicious
ed82bf6bb8 feat(ui): disable control adapter buttons if no models available 2023-10-07 22:30:15 +11:00
psychedelicious
078c9b6964 feat(nodes,ui): add t2i to linear UI
- Update backend metadata for t2i adapter
- Fix typo in `T2IAdapterInvocation`: `ip_adapter_model` -> `t2i_adapter_model`
- Update linear graphs to use t2i adapter
- Add client metadata recall for t2i adapter
- Fix bug with controlnet metadata recall - processor should be set to 'none' when recalling a control adapter
2023-10-07 22:30:15 +11:00
psychedelicious
b57ebe52e4 chore(ui): "controlnet" -> "controladapters" 2023-10-07 22:30:15 +11:00
psychedelicious
913fc83cbf fix(ui): fix control adapter autoprocess 2023-10-07 22:30:15 +11:00
psychedelicious
9508e0c9db feat(ui): refactor control adapters
Control adapters logic/state/ui is now generalized to hold controlnet, ip_adapter and t2i_adapter. In the future, other control adapter types can be added.

TODO:
- Limit IP adapter to 1
- Add T2I adapter to linear graphs
- Fix autoprocess
- T2I metadata saving & recall
- Improve on control adapters UI
2023-10-07 22:30:15 +11:00
psychedelicious
2acc93eb8e feat(ui): remove all calls to getBoardImagesTotals/getBoardAssetsTotals
This caused a crapload of network requests any time an image was generated.

The counts are necessary to handle the logic for inserting images into existing image list caches; we have to keep track of the counts.

Replace tag invalidation with manual cache updates in all cases, except the initial request (which is necessary to get the initial image counts).

One subtle change is to make the counts an object instead of a number. This is required for `immer` to handle draft states. This should be raised as a bug with RTK Query, as no error is thrown when attempting to update a primitive immer draft.
2023-10-05 08:11:52 +11:00
psychedelicious
069d8b5812 feat(ui): move initial IP adapter model selection to listener 2023-10-04 08:41:37 +11:00
psychedelicious
f002ae8da5
feat(ui): max upscale pixels config (#4765)
* feat(ui): max upscale pixels config

Add `maxUpscalePixels: number` to the app config. The number should be the *total* number of pixels eg `maxUpscalePixels: 4096 * 4096`.

If not provided, any size image may be upscaled.

If the config is provided, users will see be advised if their image is too large for either model, or told to switch to an x2 model if it's only too large for x4.

The message is via tooltip in the popover and via toast if the user uses the hotkey to upscale.

* feat(ui): "mayUpscale" -> "isAllowedToUpscale"
2023-10-02 23:25:05 +00:00
Mary Hipp
208bf68ba2 fix missing toast message 2023-10-03 07:45:26 +11:00
Mary Hipp
1aba369c83 invalidate board cache when an image is added to a board 2023-10-02 19:40:11 +11:00
chainchompa
c7f80cd163
Use metadata ip adapter (#4715)
* add control net to useRecallParams

* got recall controlnets working

* fix metadata viewer controlnet

* fix type errors

* fix controlnet metadata viewer

* add ip adapter to metadata

* added ip adapter to recall parameters

* got ip adapter recall working, still need to fix type errors

* fix type issues

* clean up logs

* python formatting

* cleanup

* fix(ui): only store `image_name` as ip adapter image

* fix(ui): use nullish coalescing operator for numbers

Need to use the nullish coalescing operator `??` instead of false-y coalescing operator `||` when the value being check is a number. This prevents unintended coalescing when the value is zero and therefore false-y.

* feat(ui): fall back on default values for ip adapter metadata

* fix(ui): remove unused schema

* feat(ui): re-use existing schemas in metadata schema

* fix(ui): do not disable invocationCache

---------

Co-authored-by: psychedelicious <4822129+psychedelicious@users.noreply.github.com>
2023-09-28 09:05:32 +00:00
psychedelicious
a0bd8c638e chore(ui): lint 2023-09-28 12:39:00 +10:00
Mary Hipp
de04a5f441 cleanup 2023-09-28 12:39:00 +10:00
Mary Hipp
40ed218c26 surface usage errors for cnet and upscale, handle clearing cnet if error occurs 2023-09-28 12:39:00 +10:00
Mary Hipp
807c6b41c5 surface usage errors for enqueuing batch 2023-09-28 12:39:00 +10:00
psychedelicious
8158124679 fix(ui): usePreselectedImage causing re-renders
This hook was rerendering any time anything changed. Moved it to a logical component, put its useEffects inside the component. This reduces the effect of the rerenders to just that tiny always-null component.
2023-09-28 09:02:45 +10:00
Mary Hipp Rogers
3432fd72f8
fix auto-switch alongside starred images (#4708)
* add skeleton loading state for queue lit

* add optional selectedImage when switching a board

* unstage

---------

Co-authored-by: Mary Hipp <maryhipp@Marys-MacBook-Air.local>
Co-authored-by: psychedelicious <4822129+psychedelicious@users.noreply.github.com>
2023-09-27 07:51:37 +00:00
blessedcoolant
b3f4f28d76 fix: Canvas pull getting cropped for Control Images 2023-09-27 12:25:45 +10:00
blessedcoolant
acee4bd282 fix: Always use bbox bounds for Controlnet Image (canvas) 2023-09-27 12:25:45 +10:00
psychedelicious
1419977e89 feat(ui): update cache status on queue event
It was polling every 5s before. No need - just invalidate the tag when we have a queue item status change event.
2023-09-27 08:56:14 +10:00
psychedelicious
a4cdaa245e
feat(ui): improve error handling (#4699)
* feat(ui): add error handling for enqueueBatch route, remove sessions

This re-implements the handling for the session create/invoke errors, but for batches.

Also remove all references to the old sessions routes in the UI.

* feat(ui): improve canvas image error UI

* make canvas error state gray instead of red

---------

Co-authored-by: Mary Hipp <maryhipp@Marys-MacBook-Air.local>
2023-09-26 15:24:53 +00:00
psychedelicious
ec19fcafb1 fix(ui): fix circular dependency
This is actually a platform-specific issue. `madge` is complaining about a circular dependency on a single file - `invokeai/frontend/web/src/features/queue/store/nanoStores.ts`. In that file, we import from the `nanostores` package. Very similar name to the file itself.

The error only appears on Windows and macOS, I imagine because those systems both resolve `nanostores` to itself before resolving to the package.

The solution is simple - rename `nanoStores.ts`. It's now `queueNanoStore.ts`.
2023-09-25 10:45:38 +10:00
psychedelicious
cba31efd78 fix(ui): do not process gallery logic for image primitive node 2023-09-23 10:02:55 +10:00
psychedelicious
88c8cb61f0 feat(ui): update linear UI to use new board field on save_image
- No longer need to make network request to add image to board after it's finished - removed
- Update linear graphs & upscale graph to save image to the board
- Update autoSwitch logic so when image is generated we still switch to the right board
2023-09-22 10:11:20 -04:00
Mary Hipp Rogers
14ce7cf09c
fix circular dep with recallAllParameters (#4640)
* break out separate functions for preselected images, remove recallAllParameters dep as it causes circular logic with model being set

* lint

---------

Co-authored-by: Mary Hipp <maryhipp@Marys-MacBook-Air.local>
2023-09-21 15:08:32 -04:00
psychedelicious
7ac99d6bc3 feat(nodes): add enable, disable, status to invocation cache
- New routes to clear, enable, disable and get the status of the cache
- Status includes hits, misses, size, max size, enabled
- Add client cache queries and mutations, abstracted into hooks
- Add invocation cache status area (next to queue status) w/ buttons
2023-09-21 09:45:52 -04:00
Dmitry Parnas
aa82f9360c fix(ui): passing Promise into ClipboardItem to make it work in Safari
throwing Error in getBaseLayerBlob, instead of returning nil
using copyBlobToClipboard for both Canvas and Text2Image clipboard functionality
2023-09-21 23:36:05 +10:00
psychedelicious
aa4b56baf2 feat(ui): enable control adapters on image drop
- Dropping/uploading an image on control adapter enables it (controlnet & ip adapter)
- The image components are always enabled to allow this
2023-09-21 14:50:55 +10:00