Add concepts for metadata handlers. Handlers include parsers, recallers and validators for different metadata types:
- Parsers parse a raw metadata object of any shape to a structured object.
- Recallers load the parsed metadata into state. Recallers are optional, as some metadata types don't need to be loaded into state.
- Validators provide an additional layer of validation before recalling the metadata. This is needed because a metadata object may be valid, but not able to be recalled due to some other requirement, like base model compatibility. Validators are optional.
Sometimes metadata is not a single object but a list of items - like LoRAs. Metadata handlers may implement an optional set of "item" handlers which operate on individual items in the list.
Parsers and validators are async to allow fetching additional data, like a model config. Recallers are synchronous.
The these handlers are composed into a public API, exported as a `handlers` object. Besides the handlers functions, a metadata handler set includes:
- A function to get the label of the metadata type.
- An optional function to render the value of the metadata type.
- An optional function to render the _item_ value of the metadata type.
Refactor of metadata recall handling. This is in preparation for a backwards compatibility layer for models.
- Create helpers to fetch a model outside react (e.g. not in a hook)
- Created helpers to parse model metadata
- Renamed a lot of types that were confusing and/or had naming collisions
- Update most model identifiers to be `{key: string}` instead of name/base/type. Doesn't change the model select components yet.
- Update model _parameters_, stored in redux, to be `{key: string, base: BaseModel}` - we need to store the base model to be able to check model compatibility. May want to store the whole config? Not sure...
We had a one-behind issue with recalling metadata items that had a model.
For example, when recalling LoRAs, we check against the current main model to decide whether or not the requested LoRA is compatible and may be recalled.
When recalling all params, we are often also recalling the main model, but the compat logic didn't compare against this new main model.
The logic is updated to check against the new main model, if one is being set.
Closes#5512
There are a few breaking changes, which I've addressed.
The vast majority of changes are related to new handling of `reselect`'s `createSelector` options.
For better or worse, we memoize just about all our selectors using lodash `isEqual` for `resultEqualityCheck`. The upgrade requires we explicitly set the `memoize` option to `lruMemoize` to continue using lodash here.
Doing that required changing our `defaultSelectorOptions`.
Instead of changing that and finding dozens of instances where we weren't using that and instead were defining selector options manually, I've created a pre-configured selector: `createMemoizedSelector`.
This is now used everywhere instead of `createSelector`.
Node authors may now create their own arbitrary/custom field types. Any pydantic model is supported.
Two notes:
1. Your field type's class name must be unique.
Suggest prefixing fields with something related to the node pack as a kind of namespace.
2. Custom field types function as connection-only fields.
For example, if your custom field has string attributes, you will not get a text input for that attribute when you give a node a field with your custom type.
This is the same behaviour as other complex fields that don't have custom UIs in the workflow editor - like, say, a string collection.
feat(ui): fix tooltips for custom types
We need to hold onto the original type of the field so they don't all just show up as "Unknown".
fix(ui): fix ts error with custom fields
feat(ui): custom field types connection validation
In the initial commit, a custom field's original type was added to the *field templates* only as `originalType`. Custom fields' `type` property was `"Custom"`*. This allowed for type safety throughout the UI logic.
*Actually, it was `"Unknown"`, but I changed it to custom for clarity.
Connection validation logic, however, uses the *field instance* of the node/field. Like the templates, *field instances* with custom types have their `type` set to `"Custom"`, but they didn't have an `originalType` property. As a result, all custom fields could be connected to all other custom fields.
To resolve this, we need to add `originalType` to the *field instances*, then switch the validation logic to use this instead of `type`.
This ended up needing a bit of fanagling:
- If we make `originalType` a required property on field instances, existing workflows will break during connection validation, because they won't have this property. We'd need a new layer of logic to migrate the workflows, adding the new `originalType` property.
While this layer is probably needed anyways, typing `originalType` as optional is much simpler. Workflow migration logic can come layer.
(Technically, we could remove all references to field types from the workflow files, and let the templates hold all this information. This feels like a significant change and I'm reluctant to do it now.)
- Because `originalType` is optional, anywhere we care about the type of a field, we need to use it over `type`. So there are a number of `field.originalType ?? field.type` expressions. This is a bit of a gotcha, we'll need to remember this in the future.
- We use `Array.prototype.includes()` often in the workflow editor, e.g. `COLLECTION_TYPES.includes(type)`. In these cases, the const array is of type `FieldType[]`, and `type` is is `FieldType`.
Because we now support custom types, the arg `type` is now widened from `FieldType` to `string`.
This causes a TS error. This behaviour is somewhat controversial (see https://github.com/microsoft/TypeScript/issues/14520). These expressions are now rewritten as `COLLECTION_TYPES.some((t) => t === type)` to satisfy TS. It's logically equivalent.
fix(ui): typo
feat(ui): add CustomCollection and CustomPolymorphic field types
feat(ui): add validation for CustomCollection & CustomPolymorphic types
- Update connection validation for custom types
- Use simple string parsing to determine if a field is a collection or polymorphic type.
- No longer need to keep a list of collection and polymorphic types.
- Added runtime checks in `baseinvocation.py` to ensure no fields are named in such a way that it could mess up the new parsing
chore(ui): remove errant console.log
fix(ui): rename 'nodes.currentConnectionFieldType' -> 'nodes.connectionStartFieldType'
This was confusingly named and kept tripping me up. Renamed to be consistent with the `reactflow` `ConnectionStartParams` type.
fix(ui): fix ts error
feat(nodes): add runtime check for custom field names
"Custom", "CustomCollection" and "CustomPolymorphic" are reserved field names.
chore(ui): add TODO for revising field type names
wip refactor fieldtype structured
wip refactor field types
wip refactor types
wip refactor types
fix node layout
refactor field types
chore: mypy
organisation
organisation
organisation
fix(nodes): fix field orig_required, field_kind and input statuses
feat(nodes): remove broken implementation of default_factory on InputField
Use of this could break connection validation due to the difference in node schemas required fields and invoke() required args.
Removed entirely for now. It wasn't ever actually used by the system, because all graphs always had values provided for fields where default_factory was used.
Also, pydantic is smart enough to not reuse the same object when specifying a default value - it clones the object first. So, the common pattern of `default_factory=list` is extraneous. It can just be `default=[]`.
fix(nodes): fix InputField name validation
workflow validation
validation
chore: ruff
feat(nodes): fix up baseinvocation comments
fix(ui): improve typing & logic of buildFieldInputTemplate
improved error handling in parseFieldType
fix: back compat for deprecated default_factory and UIType
feat(nodes): do not show node packs loaded log if none loaded
chore(ui): typegen
* working on recall height/width
* working on adding resize
* working on feature
* fix(ui): move added translation from dist/ to public/
* fix(ui): use `metadata` as hotkey cb dependency
Using `imageDTO` may result in stale data being used
---------
Co-authored-by: psychedelicious <4822129+psychedelicious@users.noreply.github.com>
* adding VAE recall when using all parameters
* adding VAE to the RecallParameters tab in ImageMetadataActions
* checking for nil vae and casting to null if undefined
* adding default VAE to recall actions list if VAE is nullish
* fix(ui): use `lodash-es` for tree-shakeable imports
---------
Co-authored-by: psychedelicious <4822129+psychedelicious@users.noreply.github.com>
* working
* added selector for method
* refactoring graph
* added ersgan method
* fixing yarn build
* add tooltips
* a conjuction
* rephrase
* removed manual sliders, set HRF to calculate dimensions automatically to match 512^2 pixels
* working
* working
* working
* fixed tooltip
* add hrf to use all parameters
* adding hrf method to parameters
* working on parameter recall
* working on parameter recall
* cleaning
* fix(ui): fix unnecessary casts in addHrfToGraph
* chore(ui): use camelCase in addHrfToGraph
* fix(ui): do not add HRF metadata unless HRF is added to graph
* fix(ui): remove unused imports in addHrfToGraph
* feat(ui): do not hide HRF params when disabled, only disable them
* fix(ui): remove unused vars in addHrfToGraph
* feat(ui): default HRF str to 0.35, method ESRGAN
* fix(ui): use isValidBoolean to check hrfEnabled param
* fix(nodes): update CoreMetadataInvocation fields for HRF
* feat(ui): set hrf strength default to 0.45
* fix(ui): set default hrf strength in configSlice
* feat(ui): use translations for HRF features
---------
Co-authored-by: psychedelicious <4822129+psychedelicious@users.noreply.github.com>
- Update backend metadata for t2i adapter
- Fix typo in `T2IAdapterInvocation`: `ip_adapter_model` -> `t2i_adapter_model`
- Update linear graphs to use t2i adapter
- Add client metadata recall for t2i adapter
- Fix bug with controlnet metadata recall - processor should be set to 'none' when recalling a control adapter
Control adapters logic/state/ui is now generalized to hold controlnet, ip_adapter and t2i_adapter. In the future, other control adapter types can be added.
TODO:
- Limit IP adapter to 1
- Add T2I adapter to linear graphs
- Fix autoprocess
- T2I metadata saving & recall
- Improve on control adapters UI
* feat(ui): max upscale pixels config
Add `maxUpscalePixels: number` to the app config. The number should be the *total* number of pixels eg `maxUpscalePixels: 4096 * 4096`.
If not provided, any size image may be upscaled.
If the config is provided, users will see be advised if their image is too large for either model, or told to switch to an x2 model if it's only too large for x4.
The message is via tooltip in the popover and via toast if the user uses the hotkey to upscale.
* feat(ui): "mayUpscale" -> "isAllowedToUpscale"
* add control net to useRecallParams
* got recall controlnets working
* fix metadata viewer controlnet
* fix type errors
* fix controlnet metadata viewer
* add ip adapter to metadata
* added ip adapter to recall parameters
* got ip adapter recall working, still need to fix type errors
* fix type issues
* clean up logs
* python formatting
* cleanup
* fix(ui): only store `image_name` as ip adapter image
* fix(ui): use nullish coalescing operator for numbers
Need to use the nullish coalescing operator `??` instead of false-y coalescing operator `||` when the value being check is a number. This prevents unintended coalescing when the value is zero and therefore false-y.
* feat(ui): fall back on default values for ip adapter metadata
* fix(ui): remove unused schema
* feat(ui): re-use existing schemas in metadata schema
* fix(ui): do not disable invocationCache
---------
Co-authored-by: psychedelicious <4822129+psychedelicious@users.noreply.github.com>
This hook was rerendering any time anything changed. Moved it to a logical component, put its useEffects inside the component. This reduces the effect of the rerenders to just that tiny always-null component.
* add control net to useRecallParams
* got recall controlnets working
* fix metadata viewer controlnet
* fix type errors
* fix controlnet metadata viewer
* set control image and use correct processor type and node
* clean up logs
* recall processor using substring
* feat(ui): enable controlNet when recalling one
---------
Co-authored-by: psychedelicious <4822129+psychedelicious@users.noreply.github.com>
* break out separate functions for preselected images, remove recallAllParameters dep as it causes circular logic with model being set
* lint
---------
Co-authored-by: Mary Hipp <maryhipp@Marys-MacBook-Air.local>
multi-select actions include:
- drag to board to move all to that board
- right click to add all to board or delete all
backend changes:
- add routes for changing board for list of image names, deleting list of images
- change image-specific routes to `images/i/{image_name}` to not clobber other routes (like `images/upload`, `images/delete`)
- subclass pydantic `BaseModel` as `BaseModelExcludeNull`, which excludes null values when calling `dict()` on the model. this fixes inconsistent types related to JSON parsing null values into `null` instead of `undefined`
- remove `board_id` from `remove_image_from_board`
frontend changes:
- multi-selection stuff uses `ImageDTO[]` as payloads, for dnd and other mutations. this gives us access to image `board_id`s when hitting routes, and enables efficient cache updates.
- consolidate change board and delete image modals to handle single and multiples
- board totals are now re-fetched on mutation and not kept in sync manually - was way too tedious to do this
- fixed warning about nested `<p>` elements
- closes#4088 , need to handle case when `autoAddBoardId` is `"none"`
- add option to show gallery image delete button on every gallery image
frontend refactors/organisation:
- make typegen script js instead of ts
- enable `noUncheckedIndexedAccess` to help avoid bugs when indexing into arrays, many small changes needed to satisfy TS after this
- move all image-related endpoints into `endpoints/images.ts`, its a big file now, but this fixes a number of circular dependency issues that were otherwise felt impossible to resolve
