Commit Graph

580 Commits

Author SHA1 Message Date
Ryan Dick
f7f697849c Skip weight initialization when resizing text encoder token embeddings to accomodate new TI embeddings. This saves time. 2024-01-05 15:16:00 -05:00
Millun Atluri
53b835945f Updated with ruff formatting 2023-12-28 11:05:19 +11:00
woweenie
e38d0e39b7
fix bug when there are two multi vector TI in a prompt 2023-12-27 22:14:14 +01:00
Ryan Dick
cb698ff1fb Update model_probe to work with diffuser-format SD TI embeddings. 2023-12-18 09:51:16 -05:00
Lincoln Stein
212dbaf9a2 fix comment 2023-12-15 00:25:27 -05:00
Lincoln Stein
ac3cf48d7f make probe recognize lora format at https://civitai.com/models/224641 2023-12-15 00:25:27 -05:00
Lincoln Stein
75089b7a9d merge in changes from main 2023-12-01 09:18:07 -05:00
Lincoln Stein
5a3f1f2b22 fix ruff github format errors 2023-12-01 01:59:26 -05:00
Lincoln Stein
f95ce1870c fix ruff format check 2023-12-01 01:46:12 -05:00
Lincoln Stein
0719a46372 add support for SDXL textual inversion/embeddings 2023-12-01 01:28:28 -05:00
Lincoln Stein
ecd3dcd5df
Merge branch 'main' into refactor/model-manager-3 2023-11-27 22:15:51 -05:00
Steven Frank
e509d719ee Fix attempt to deserialize on CUDA on Mac
Without specifying "cpu", attempts to use non-existent CUDA to deserialize embeddings on macOS, resulting in a warning / failure to load.
2023-11-28 09:24:57 +11:00
Lincoln Stein
8ef596eac7 further changes for ruff 2023-11-26 17:13:31 -05:00
Lincoln Stein
8c7a7bc897 Merge branch 'main' into refactor/model-manager-3 2023-11-22 22:29:23 -05:00
Lincoln Stein
4aab728590 move name/description logic into model_probe.py 2023-11-22 22:29:02 -05:00
Lincoln Stein
98a4930a52 add probe support for LCM main models 2023-11-22 14:58:27 -05:00
psychedelicious
1a596a5684 fix(backend): fix unintentional change to import orders
- Ignore I001 (isort rules) for this file
- Ignore F401 (unused imports) for this file
2023-11-21 20:22:27 +11:00
psychedelicious
da443973cb chore: ruff 2023-11-21 20:22:27 +11:00
psychedelicious
6494e8e551 chore: ruff format 2023-11-11 10:55:40 +11:00
psychedelicious
99a8ebe3a0 chore: ruff check - fix flake8-bugbear 2023-11-11 10:55:28 +11:00
psychedelicious
3a136420d5 chore: ruff check - fix flake8-comprensions 2023-11-11 10:55:23 +11:00
Wubbbi
6001d3d71d Change pad_to_multiple_of to be 8 for all cases. Add comment about it's temporary status 2023-11-10 17:51:59 -05:00
Wubbbi
8831d1ee41 Update Documentation 2023-11-10 17:51:59 -05:00
Wubbbi
a0be83e370 Update Transformers to 4.34 and fix pad_to_multiple_of 2023-11-10 17:51:59 -05:00
Lincoln Stein
8702a63197 add support for downloading and installing LCM lora diffusers models 2023-11-10 17:51:30 -05:00
psychedelicious
6aa87f973e fix(nodes): create app/shared/ module to prevent circular imports
We have a number of shared classes, objects, and functions that are used in multiple places. This causes circular import issues.

This commit creates a new `app/shared/` module to hold these shared classes, objects, and functions.

Initially, only `FreeUConfig` and `FieldDescriptions` are moved here. This resolves a circular import issue with custom nodes.

Other shared classes, objects, and functions will be moved here in future commits.
2023-11-09 16:41:55 +11:00
Kent Keirsey
e66d0f7372
Merge branch 'main' into feat/nodes/freeu 2023-11-06 05:39:58 -08:00
Ryan Dick
aa02ebf8f5 Fix model cache gc.collect() condition. 2023-11-04 08:52:10 -04:00
Ryan Dick
fb3d0c4b12 Fix bug in model cache reference count checking. 2023-11-03 13:50:40 -07:00
Ryan Dick
8488ab0134 Reduce frequency that we call gc.collect() in the model cache. 2023-11-03 13:50:40 -07:00
Ryan Dick
875231ed3d Add reminder to clean up our model cache clearing logic. 2023-11-03 13:50:40 -07:00
Ryan Dick
43b300498f Remove explicit gc.collect() after transferring models from device to CPU. I'm not sure why this was there in the first place, but it was taking a significant amount of time (up to ~1sec in my tests). 2023-11-03 13:50:40 -07:00
Ryan Dick
e391f3c9a8 Skip torch.nn.Embedding.reset_parameters(...) when loading a text encoder model. 2023-11-02 19:41:33 -07:00
Ryan Dick
6e7a3f0546 (minor) Fix static checks and typo. 2023-11-02 19:20:37 -07:00
Ryan Dick
4a683cc669 Add a app config parameter to control the ModelCache logging behavior. 2023-11-02 19:20:37 -07:00
Ryan Dick
3781e56e57 Add log_memory_usage param to ModelCache. 2023-11-02 19:20:37 -07:00
Ryan Dick
8ff49109a8 Update get_pretty_snapshot_diff(...) to handle None-snapshots. 2023-11-02 19:20:37 -07:00
Ryan Dick
bac2a757e8 Replace deepcopy with a pickle roundtrip in apply_ti(...) to improve speed. 2023-11-02 19:05:24 -07:00
Ryan Dick
fa7f6a6a10 Further tidying of LoRA patching. Revert some changes that didn't end up being important under the constraint that calculations are done on the same device as the model. 2023-11-02 10:03:17 -07:00
Ryan Dick
61b17c475a Add TODO note about improving _resolve_lora_key(...). 2023-11-02 10:03:17 -07:00
Ryan Dick
379d68f595 Patch LoRA on device when model is already on device. 2023-11-02 10:03:17 -07:00
Ryan Dick
545c811bf1 Remove device and dtype members from LoRAModelRaw, they can too easily get out-of-sync with the underlying layer states. 2023-11-02 10:03:17 -07:00
Ryan Dick
2ba5b44ec4 Remove unused _lora_forward_hook(...). 2023-11-02 10:03:17 -07:00
Ryan Dick
7f4ce518b7 auto-format lora.py 2023-11-02 10:03:17 -07:00
Lincoln Stein
6cbc69f3b7 support conversion of controlnets from safetensors to diffusers 2023-10-23 22:06:10 -04:00
d8ahazard
fdf02c33d0 Catch generic model errors
Prevent the app from dying on invalid models.
2023-10-19 07:28:33 +11:00
Millun Atluri
001bba1719
Merge branch 'main' into feat/nodes/freeu 2023-10-17 15:58:00 +11:00
psychedelicious
c238a7f18b feat(api): chore: pydantic & fastapi upgrade
Upgrade pydantic and fastapi to latest.

- pydantic~=2.4.2
- fastapi~=103.2
- fastapi-events~=0.9.1

**Big Changes**

There are a number of logic changes needed to support pydantic v2. Most changes are very simple, like using the new methods to serialized and deserialize models, but there are a few more complex changes.

**Invocations**

The biggest change relates to invocation creation, instantiation and validation.

Because pydantic v2 moves all validation logic into the rust pydantic-core, we may no longer directly stick our fingers into the validation pie.

Previously, we (ab)used models and fields to allow invocation fields to be optional at instantiation, but required when `invoke()` is called. We directly manipulated the fields and invocation models when calling `invoke()`.

With pydantic v2, this is much more involved. Changes to the python wrapper do not propagate down to the rust validation logic - you have to rebuild the model. This causes problem with concurrent access to the invocation classes and is not a free operation.

This logic has been totally refactored and we do not need to change the model any more. The details are in `baseinvocation.py`, in the `InputField` function and `BaseInvocation.invoke_internal()` method.

In the end, this implementation is cleaner.

**Invocation Fields**

In pydantic v2, you can no longer directly add or remove fields from a model.

Previously, we did this to add the `type` field to invocations.

**Invocation Decorators**

With pydantic v2, we instead use the imperative `create_model()` API to create a new model with the additional field. This is done in `baseinvocation.py` in the `invocation()` wrapper.

A similar technique is used for `invocation_output()`.

**Minor Changes**

There are a number of minor changes around the pydantic v2 models API.

**Protected `model_` Namespace**

All models' pydantic-provided methods and attributes are prefixed with `model_` and this is considered a protected namespace. This causes some conflict, because "model" means something to us, and we have a ton of pydantic models with attributes starting with "model_".

Forunately, there are no direct conflicts. However, in any pydantic model where we define an attribute or method that starts with "model_", we must tell set the protected namespaces to an empty tuple.

```py
class IPAdapterModelField(BaseModel):
    model_name: str = Field(description="Name of the IP-Adapter model")
    base_model: BaseModelType = Field(description="Base model")

    model_config = ConfigDict(protected_namespaces=())
```

**Model Serialization**

Pydantic models no longer have `Model.dict()` or `Model.json()`.

Instead, we use `Model.model_dump()` or `Model.model_dump_json()`.

**Model Deserialization**

Pydantic models no longer have `Model.parse_obj()` or `Model.parse_raw()`, and there are no `parse_raw_as()` or `parse_obj_as()` functions.

Instead, you need to create a `TypeAdapter` object to parse python objects or JSON into a model.

```py
adapter_graph = TypeAdapter(Graph)
deserialized_graph_from_json = adapter_graph.validate_json(graph_json)
deserialized_graph_from_dict = adapter_graph.validate_python(graph_dict)
```

**Field Customisation**

Pydantic `Field`s no longer accept arbitrary args.

Now, you must put all additional arbitrary args in a `json_schema_extra` arg on the field.

**Schema Customisation**

FastAPI and pydantic schema generation now follows the OpenAPI version 3.1 spec.

This necessitates two changes:
- Our schema customization logic has been revised
- Schema parsing to build node templates has been revised

The specific aren't important, but this does present additional surface area for bugs.

**Performance Improvements**

Pydantic v2 is a full rewrite with a rust backend. This offers a substantial performance improvement (pydantic claims 5x to 50x depending on the task). We'll notice this the most during serialization and deserialization of sessions/graphs, which happens very very often - a couple times per node.

I haven't done any benchmarks, but anecdotally, graph execution is much faster. Also, very larges graphs - like with massive iterators - are much, much faster.
2023-10-17 14:59:25 +11:00
blessedcoolant
c9d95e5758
Merge branch 'main' into bugfix/ignore-dot-directories-on-model-scan 2023-10-15 17:23:02 +05:30
psychedelicious
48626c40fd fix(backend): handle systems with glibc < 2.33
`mallinfo2` is not available on `glibc` < 2.33.

On these systems, we successfully load the library but get an `AttributeError` on attempting to access `mallinfo2`.

I'm not sure if the old `mallinfo` will work, and not sure how to install it safely to test, so for now we just handle the `AttributeError`.

This means the enhanced memory snapshot logic will be skipped for these systems, which isn't a big deal.
2023-10-15 07:56:55 +11:00