feat(api): chore: pydantic & fastapi upgrade

Upgrade pydantic and fastapi to latest.

- pydantic~=2.4.2
- fastapi~=103.2
- fastapi-events~=0.9.1

**Big Changes**

There are a number of logic changes needed to support pydantic v2. Most changes are very simple, like using the new methods to serialized and deserialize models, but there are a few more complex changes.

**Invocations**

The biggest change relates to invocation creation, instantiation and validation.

Because pydantic v2 moves all validation logic into the rust pydantic-core, we may no longer directly stick our fingers into the validation pie.

Previously, we (ab)used models and fields to allow invocation fields to be optional at instantiation, but required when `invoke()` is called. We directly manipulated the fields and invocation models when calling `invoke()`.

With pydantic v2, this is much more involved. Changes to the python wrapper do not propagate down to the rust validation logic - you have to rebuild the model. This causes problem with concurrent access to the invocation classes and is not a free operation.

This logic has been totally refactored and we do not need to change the model any more. The details are in `baseinvocation.py`, in the `InputField` function and `BaseInvocation.invoke_internal()` method.

In the end, this implementation is cleaner.

**Invocation Fields**

In pydantic v2, you can no longer directly add or remove fields from a model.

Previously, we did this to add the `type` field to invocations.

**Invocation Decorators**

With pydantic v2, we instead use the imperative `create_model()` API to create a new model with the additional field. This is done in `baseinvocation.py` in the `invocation()` wrapper.

A similar technique is used for `invocation_output()`.

**Minor Changes**

There are a number of minor changes around the pydantic v2 models API.

**Protected `model_` Namespace**

All models' pydantic-provided methods and attributes are prefixed with `model_` and this is considered a protected namespace. This causes some conflict, because "model" means something to us, and we have a ton of pydantic models with attributes starting with "model_".

Forunately, there are no direct conflicts. However, in any pydantic model where we define an attribute or method that starts with "model_", we must tell set the protected namespaces to an empty tuple.

```py
class IPAdapterModelField(BaseModel):
    model_name: str = Field(description="Name of the IP-Adapter model")
    base_model: BaseModelType = Field(description="Base model")

    model_config = ConfigDict(protected_namespaces=())
```

**Model Serialization**

Pydantic models no longer have `Model.dict()` or `Model.json()`.

Instead, we use `Model.model_dump()` or `Model.model_dump_json()`.

**Model Deserialization**

Pydantic models no longer have `Model.parse_obj()` or `Model.parse_raw()`, and there are no `parse_raw_as()` or `parse_obj_as()` functions.

Instead, you need to create a `TypeAdapter` object to parse python objects or JSON into a model.

```py
adapter_graph = TypeAdapter(Graph)
deserialized_graph_from_json = adapter_graph.validate_json(graph_json)
deserialized_graph_from_dict = adapter_graph.validate_python(graph_dict)
```

**Field Customisation**

Pydantic `Field`s no longer accept arbitrary args.

Now, you must put all additional arbitrary args in a `json_schema_extra` arg on the field.

**Schema Customisation**

FastAPI and pydantic schema generation now follows the OpenAPI version 3.1 spec.

This necessitates two changes:
- Our schema customization logic has been revised
- Schema parsing to build node templates has been revised

The specific aren't important, but this does present additional surface area for bugs.

**Performance Improvements**

Pydantic v2 is a full rewrite with a rust backend. This offers a substantial performance improvement (pydantic claims 5x to 50x depending on the task). We'll notice this the most during serialization and deserialization of sessions/graphs, which happens very very often - a couple times per node.

I haven't done any benchmarks, but anecdotally, graph execution is much faster. Also, very larges graphs - like with massive iterators - are much, much faster.

tests/nodes/test_node_graph.py

@@ -1,4 +1,5 @@
 import pytest
+from pydantic import TypeAdapter
 
 from invokeai.app.invocations.baseinvocation import (
     BaseInvocation,
@@ -593,20 +594,21 @@ def test_graph_can_serialize():
     g.add_edge(e)
 
     # Not throwing on this line is sufficient
-    _ = g.json()
+    _ = g.model_dump_json()
 
 
 def test_graph_can_deserialize():
     g = Graph()
     n1 = TextToImageTestInvocation(id="1", prompt="Banana sushi")
-    n2 = ESRGANInvocation(id="2")
+    n2 = ImageToImageTestInvocation(id="2")
     g.add_node(n1)
     g.add_node(n2)
     e = create_edge(n1.id, "image", n2.id, "image")
     g.add_edge(e)
 
-    json = g.json()
-    g2 = Graph.parse_raw(json)
+    json = g.model_dump_json()
+    adapter_graph = TypeAdapter(Graph)
+    g2 = adapter_graph.validate_json(json)
 
     assert g2 is not None
     assert g2.nodes["1"] is not None
@@ -619,7 +621,7 @@ def test_graph_can_deserialize():
 
 
 def test_invocation_decorator():
-    invocation_type = "test_invocation"
+    invocation_type = "test_invocation_decorator"
     title = "Test Invocation"
     tags = ["first", "second", "third"]
     category = "category"
@@ -630,7 +632,7 @@ def test_invocation_decorator():
         def invoke(self):
             pass
 
-    schema = TestInvocation.schema()
+    schema = TestInvocation.model_json_schema()
 
     assert schema.get("title") == title
     assert schema.get("tags") == tags
@@ -640,18 +642,17 @@ def test_invocation_decorator():
 
 
 def test_invocation_version_must_be_semver():
-    invocation_type = "test_invocation"
     valid_version = "1.0.0"
     invalid_version = "not_semver"
 
-    @invocation(invocation_type, version=valid_version)
+    @invocation("test_invocation_version_valid", version=valid_version)
     class ValidVersionInvocation(BaseInvocation):
         def invoke(self):
             pass
 
     with pytest.raises(InvalidVersionError):
 
-        @invocation(invocation_type, version=invalid_version)
+        @invocation("test_invocation_version_invalid", version=invalid_version)
         class InvalidVersionInvocation(BaseInvocation):
             def invoke(self):
                 pass
@@ -694,4 +695,4 @@ def test_ints_do_not_accept_floats():
 def test_graph_can_generate_schema():
     # Not throwing on this line is sufficient
     # NOTE: if this test fails, it's PROBABLY because a new invocation type is breaking schema generation
-    _ = Graph.schema_json(indent=2)
+    _ = Graph.model_json_schema()

tests/nodes/test_session_queue.py

@@ -1,5 +1,5 @@
 import pytest
-from pydantic import ValidationError, parse_raw_as
+from pydantic import TypeAdapter, ValidationError
 
 from invokeai.app.services.session_queue.session_queue_common import (
     Batch,
@@ -150,8 +150,9 @@ def test_prepare_values_to_insert(batch_data_collection, batch_graph):
     values = prepare_values_to_insert(queue_id="default", batch=b, priority=0, max_new_queue_items=1000)
     assert len(values) == 8
 
+    session_adapter = TypeAdapter(GraphExecutionState)
     # graph should be serialized
-    ges = parse_raw_as(GraphExecutionState, values[0].session)
+    ges = session_adapter.validate_json(values[0].session)
 
     # graph values should be populated
     assert ges.graph.get_node("1").prompt == "Banana sushi"
@@ -160,15 +161,16 @@ def test_prepare_values_to_insert(batch_data_collection, batch_graph):
     assert ges.graph.get_node("4").prompt == "Nissan"
 
     # session ids should match deserialized graph
-    assert [v.session_id for v in values] == [parse_raw_as(GraphExecutionState, v.session).id for v in values]
+    assert [v.session_id for v in values] == [session_adapter.validate_json(v.session).id for v in values]
 
     # should unique session ids
     sids = [v.session_id for v in values]
     assert len(sids) == len(set(sids))
 
+    nfv_list_adapter = TypeAdapter(list[NodeFieldValue])
     # should have 3 node field values
     assert type(values[0].field_values) is str
-    assert len(parse_raw_as(list[NodeFieldValue], values[0].field_values)) == 3
+    assert len(nfv_list_adapter.validate_json(values[0].field_values)) == 3
 
     # should have batch id and priority
     assert all(v.batch_id == b.batch_id for v in values)

tests/nodes/test_sqlite.py

@@ -15,7 +15,8 @@ class TestModel(BaseModel):
 @pytest.fixture
 def db() -> SqliteItemStorage[TestModel]:
     sqlite_db = SqliteDatabase(InvokeAIAppConfig(use_memory_db=True), InvokeAILogger.get_logger())
-    return SqliteItemStorage[TestModel](db=sqlite_db, table_name="test", id_field="id")
+    sqlite_item_storage = SqliteItemStorage[TestModel](db=sqlite_db, table_name="test", id_field="id")
+    return sqlite_item_storage
 
 
 def test_sqlite_service_can_create_and_get(db: SqliteItemStorage[TestModel]):