- Support grid size of 8 on canvas
- Internal canvas math works on 8
- Update gridlines rendering to show 64 spaced lines and 32/16/8 when zoomed in
- Bbox manipulation defaults to grid of 64 - hold shift to get grid of 8
Besides being something we support internally, supporting 8 on canvas avoids a lot of hacky logic needed to work well with aspect ratios.
Canvas and non-canvas have separate width and height and need their own separate aspect ratios. In order to not duplicate a lot of aspect ratio logic, the components relating to image size have been modularized.
There are a few breaking changes, which I've addressed.
The vast majority of changes are related to new handling of `reselect`'s `createSelector` options.
For better or worse, we memoize just about all our selectors using lodash `isEqual` for `resultEqualityCheck`. The upgrade requires we explicitly set the `memoize` option to `lruMemoize` to continue using lodash here.
Doing that required changing our `defaultSelectorOptions`.
Instead of changing that and finding dozens of instances where we weren't using that and instead were defining selector options manually, I've created a pre-configured selector: `createMemoizedSelector`.
This is now used everywhere instead of `createSelector`.
- Reset init image, control adapter images, and node image fields when their selected image fails to load
- Only do this if the app is connected via socket (this indicates that the image is "really" gone, and there isn't just a transient network issue)
It's possible for image parameters/nodes/states to have reference a deleted image. For example, a resize image node might have an image set on it, and the workflow saved. The workflow contains a hard reference to that image.
The image is deleted and the workflow loaded again later. The deleted image is still in that workflow, but the app doesn't detect that. The result is that the workflow/graph appears to be valid, but will fail on invoke.
This creates a really confusing user experience, where when somebody shares a workflow with an image baked into it, and another person opens it, everything *looks* ok, but the workflow fails with a mysterious error about a missing image.
The problem affects node images, control adapter images and the img2img init image. Resetting the image when it fails to load *and* socket is connected resolves this in a simple way.
The problem also affects canvas images, but we have handle that by displaying an error fallback image, so no change is made there.
Node authors may now create their own arbitrary/custom field types. Any pydantic model is supported.
Two notes:
1. Your field type's class name must be unique.
Suggest prefixing fields with something related to the node pack as a kind of namespace.
2. Custom field types function as connection-only fields.
For example, if your custom field has string attributes, you will not get a text input for that attribute when you give a node a field with your custom type.
This is the same behaviour as other complex fields that don't have custom UIs in the workflow editor - like, say, a string collection.
feat(ui): fix tooltips for custom types
We need to hold onto the original type of the field so they don't all just show up as "Unknown".
fix(ui): fix ts error with custom fields
feat(ui): custom field types connection validation
In the initial commit, a custom field's original type was added to the *field templates* only as `originalType`. Custom fields' `type` property was `"Custom"`*. This allowed for type safety throughout the UI logic.
*Actually, it was `"Unknown"`, but I changed it to custom for clarity.
Connection validation logic, however, uses the *field instance* of the node/field. Like the templates, *field instances* with custom types have their `type` set to `"Custom"`, but they didn't have an `originalType` property. As a result, all custom fields could be connected to all other custom fields.
To resolve this, we need to add `originalType` to the *field instances*, then switch the validation logic to use this instead of `type`.
This ended up needing a bit of fanagling:
- If we make `originalType` a required property on field instances, existing workflows will break during connection validation, because they won't have this property. We'd need a new layer of logic to migrate the workflows, adding the new `originalType` property.
While this layer is probably needed anyways, typing `originalType` as optional is much simpler. Workflow migration logic can come layer.
(Technically, we could remove all references to field types from the workflow files, and let the templates hold all this information. This feels like a significant change and I'm reluctant to do it now.)
- Because `originalType` is optional, anywhere we care about the type of a field, we need to use it over `type`. So there are a number of `field.originalType ?? field.type` expressions. This is a bit of a gotcha, we'll need to remember this in the future.
- We use `Array.prototype.includes()` often in the workflow editor, e.g. `COLLECTION_TYPES.includes(type)`. In these cases, the const array is of type `FieldType[]`, and `type` is is `FieldType`.
Because we now support custom types, the arg `type` is now widened from `FieldType` to `string`.
This causes a TS error. This behaviour is somewhat controversial (see https://github.com/microsoft/TypeScript/issues/14520). These expressions are now rewritten as `COLLECTION_TYPES.some((t) => t === type)` to satisfy TS. It's logically equivalent.
fix(ui): typo
feat(ui): add CustomCollection and CustomPolymorphic field types
feat(ui): add validation for CustomCollection & CustomPolymorphic types
- Update connection validation for custom types
- Use simple string parsing to determine if a field is a collection or polymorphic type.
- No longer need to keep a list of collection and polymorphic types.
- Added runtime checks in `baseinvocation.py` to ensure no fields are named in such a way that it could mess up the new parsing
chore(ui): remove errant console.log
fix(ui): rename 'nodes.currentConnectionFieldType' -> 'nodes.connectionStartFieldType'
This was confusingly named and kept tripping me up. Renamed to be consistent with the `reactflow` `ConnectionStartParams` type.
fix(ui): fix ts error
feat(nodes): add runtime check for custom field names
"Custom", "CustomCollection" and "CustomPolymorphic" are reserved field names.
chore(ui): add TODO for revising field type names
wip refactor fieldtype structured
wip refactor field types
wip refactor types
wip refactor types
fix node layout
refactor field types
chore: mypy
organisation
organisation
organisation
fix(nodes): fix field orig_required, field_kind and input statuses
feat(nodes): remove broken implementation of default_factory on InputField
Use of this could break connection validation due to the difference in node schemas required fields and invoke() required args.
Removed entirely for now. It wasn't ever actually used by the system, because all graphs always had values provided for fields where default_factory was used.
Also, pydantic is smart enough to not reuse the same object when specifying a default value - it clones the object first. So, the common pattern of `default_factory=list` is extraneous. It can just be `default=[]`.
fix(nodes): fix InputField name validation
workflow validation
validation
chore: ruff
feat(nodes): fix up baseinvocation comments
fix(ui): improve typing & logic of buildFieldInputTemplate
improved error handling in parseFieldType
fix: back compat for deprecated default_factory and UIType
feat(nodes): do not show node packs loaded log if none loaded
chore(ui): typegen
* working on recall height/width
* working on adding resize
* working on feature
* fix(ui): move added translation from dist/ to public/
* fix(ui): use `metadata` as hotkey cb dependency
Using `imageDTO` may result in stale data being used
---------
Co-authored-by: psychedelicious <4822129+psychedelicious@users.noreply.github.com>
* eslint added and new string added
* strings and translation hook added
* more changes made
* missing translation added
* final errors resolve in progress
* all errors resolved
* fix(ui): fix missing import of `t()`
* fix(ui): use plurals for moving images to board translation
* fix(ui): fix typo in translation key
* fix(ui): do not use translation for "invoke ai"
* chore(ui): lint
---------
Co-authored-by: psychedelicious <4822129+psychedelicious@users.noreply.github.com>
* first string only to test
* more strings changed
* almost half strings added in json file
* more strings added
* more changes
* few strings and t function changed
* resolved
* errors resolved
* chore(ui): fmt en.json
---------
Co-authored-by: psychedelicious <4822129+psychedelicious@users.noreply.github.com>
This rule enforces no arrow functions in component props. In practice, it means all functions passed as component props must be wrapped in `useCallback()`.
This is a performance optimization to prevent unnecessary rerenders.
The rule is added and all violations have been fixed, whew!
* adding VAE recall when using all parameters
* adding VAE to the RecallParameters tab in ImageMetadataActions
* checking for nil vae and casting to null if undefined
* adding default VAE to recall actions list if VAE is nullish
* fix(ui): use `lodash-es` for tree-shakeable imports
---------
Co-authored-by: psychedelicious <4822129+psychedelicious@users.noreply.github.com>
* working
* added selector for method
* refactoring graph
* added ersgan method
* fixing yarn build
* add tooltips
* a conjuction
* rephrase
* removed manual sliders, set HRF to calculate dimensions automatically to match 512^2 pixels
* working
* working
* working
* fixed tooltip
* add hrf to use all parameters
* adding hrf method to parameters
* working on parameter recall
* working on parameter recall
* cleaning
* fix(ui): fix unnecessary casts in addHrfToGraph
* chore(ui): use camelCase in addHrfToGraph
* fix(ui): do not add HRF metadata unless HRF is added to graph
* fix(ui): remove unused imports in addHrfToGraph
* feat(ui): do not hide HRF params when disabled, only disable them
* fix(ui): remove unused vars in addHrfToGraph
* feat(ui): default HRF str to 0.35, method ESRGAN
* fix(ui): use isValidBoolean to check hrfEnabled param
* fix(nodes): update CoreMetadataInvocation fields for HRF
* feat(ui): set hrf strength default to 0.45
* fix(ui): set default hrf strength in configSlice
* feat(ui): use translations for HRF features
---------
Co-authored-by: psychedelicious <4822129+psychedelicious@users.noreply.github.com>
* added HrfScale type with initial value
* working
* working
* working
* working
* working
* added addHrfToGraph
* continueing to implement this
* working on this
* comments
* working
* made hrf into its own collapse
* working on adding strength slider
* working
* working
* refactoring
* working
* change of this working: 0
* removed onnx support since apparently its not used
* working
* made scale integer
* trying out psycicpebbles idea
* working
* working on this
* working
* added toggle
* comments
* self review
* fixing things
* remove 'any' type
* fixing typing
* changed initial strength value to 3 (large values cause issues)
* set denoising start to be 1 - strength to resemble image to image
* set initial value
* added image to image
* pr1
* pr2
* updating to resolution finding
* working
* working
* working
* working
* working
* working
* working
* working
* working
* use memo
* connect rescale hw to noise
* working
* fixed min bug
* nit
* hides elements conditionally
* style
* feat(ui): add config for HRF, disable if feature disabled or ONNX model in use
* fix(ui): use `useCallback` for HRF toggle
---------
Co-authored-by: psychedelicious <4822129+psychedelicious@users.noreply.github.com>
- Update backend metadata for t2i adapter
- Fix typo in `T2IAdapterInvocation`: `ip_adapter_model` -> `t2i_adapter_model`
- Update linear graphs to use t2i adapter
- Add client metadata recall for t2i adapter
- Fix bug with controlnet metadata recall - processor should be set to 'none' when recalling a control adapter
Control adapters logic/state/ui is now generalized to hold controlnet, ip_adapter and t2i_adapter. In the future, other control adapter types can be added.
TODO:
- Limit IP adapter to 1
- Add T2I adapter to linear graphs
- Fix autoprocess
- T2I metadata saving & recall
- Improve on control adapters UI
* Bump diffusers to 0.21.2.
* Add T2IAdapterInvocation boilerplate.
* Add T2I-Adapter model to model-management.
* (minor) Tidy prepare_control_image(...).
* Add logic to run the T2I-Adapter models at the start of the DenoiseLatentsInvocation.
* Add logic for applying T2I-Adapter weights and accumulating.
* Add T2IAdapter to MODEL_CLASSES map.
* yarn typegen
* Add model probes for T2I-Adapter models.
* Add all of the frontend boilerplate required to use T2I-Adapter in the nodes editor.
* Add T2IAdapterModel.convert_if_required(...).
* Fix errors in T2I-Adapter input image sizing logic.
* Fix bug with handling of multiple T2I-Adapters.
* black / flake8
* Fix typo
* yarn build
* Add num_channels param to prepare_control_image(...).
* Link to upstream diffusers bugfix PR that currently requires a workaround.
* feat: Add Color Map Preprocessor
Needed for the color T2I Adapter
* feat: Add Color Map Preprocessor to Linear UI
* Revert "feat: Add Color Map Preprocessor"
This reverts commit a1119a00bf.
* Revert "feat: Add Color Map Preprocessor to Linear UI"
This reverts commit bd8a9b82d8.
* Fix T2I-Adapter field rendering in workflow editor.
* yarn build, yarn typegen
---------
Co-authored-by: blessedcoolant <54517381+blessedcoolant@users.noreply.github.com>
Co-authored-by: psychedelicious <4822129+psychedelicious@users.noreply.github.com>
* feat(ui): max upscale pixels config
Add `maxUpscalePixels: number` to the app config. The number should be the *total* number of pixels eg `maxUpscalePixels: 4096 * 4096`.
If not provided, any size image may be upscaled.
If the config is provided, users will see be advised if their image is too large for either model, or told to switch to an x2 model if it's only too large for x4.
The message is via tooltip in the popover and via toast if the user uses the hotkey to upscale.
* feat(ui): "mayUpscale" -> "isAllowedToUpscale"
* add control net to useRecallParams
* got recall controlnets working
* fix metadata viewer controlnet
* fix type errors
* fix controlnet metadata viewer
* add ip adapter to metadata
* added ip adapter to recall parameters
* got ip adapter recall working, still need to fix type errors
* fix type issues
* clean up logs
* python formatting
* cleanup
* fix(ui): only store `image_name` as ip adapter image
* fix(ui): use nullish coalescing operator for numbers
Need to use the nullish coalescing operator `??` instead of false-y coalescing operator `||` when the value being check is a number. This prevents unintended coalescing when the value is zero and therefore false-y.
* feat(ui): fall back on default values for ip adapter metadata
* fix(ui): remove unused schema
* feat(ui): re-use existing schemas in metadata schema
* fix(ui): do not disable invocationCache
---------
Co-authored-by: psychedelicious <4822129+psychedelicious@users.noreply.github.com>
This hook was rerendering any time anything changed. Moved it to a logical component, put its useEffects inside the component. This reduces the effect of the rerenders to just that tiny always-null component.
* add control net to useRecallParams
* got recall controlnets working
* fix metadata viewer controlnet
* fix type errors
* fix controlnet metadata viewer
* set control image and use correct processor type and node
* clean up logs
* recall processor using substring
* feat(ui): enable controlNet when recalling one
---------
Co-authored-by: psychedelicious <4822129+psychedelicious@users.noreply.github.com>
- Change translations to use arrays of paragraphs instead of a single paragraph.
- Change component to accept a `feature` prop to identify the feature which the popover describes.
- Add optional `wrapperProps`: passed to the wrapper element, allowing more flexibility when using the popover
- Add optional `popoverProps`: passed to the `<Popover />` component, allowing for overriding individual instances of the popover's props
- Move definitions of features and popover settings to `invokeai/frontend/web/src/common/components/IAIInformationalPopover/constants.ts`
- Add some type safety to the `feature` prop
- Edit `POPOVER_DATA` to provide `image`, `href`, `buttonLabel`, and any popover props. The popover props are applied to all instances of the popover for the given feature. Note that the component prop `popoverProps` will override settings here.
- Remove the popover's arrow. Because the popover is wrapping groups of components, sometimes the error ends up pointing to nothing, which looks kinda janky. I've just removed the arrow entirely, but feel free to add it back if you think it looks better.
- Use a `link` variant button with external link icon to better communicate that clicking the button will open a new tab.
- Default the link button label to "Learn More" (if a label is provided, that will be used instead)
- Make default position `top`, but set manually set some to `right` - namely, anything with a dropdown. This prevents the popovers from obscuring or being obscured by the dropdowns.
- Do a bit more restructuring of the Popover component itself, and how it is integrated with other components
- More ref forwarding
- Make the open delay 1s
- Set the popovers to use lazy mounting (eg do not mount until the user opens the thing)
- Update the verbiage for many popover items and add missing dynamic prompts stuff
* break out separate functions for preselected images, remove recallAllParameters dep as it causes circular logic with model being set
* lint
---------
Co-authored-by: Mary Hipp <maryhipp@Marys-MacBook-Air.local>
