[BUG] Null fields dropped from HTTP request bodies

Bug Report Checklist

• Have you provided a full/minimal spec to reproduce the issue?
• Have you validated the input using an OpenAPI validator (example)?
• Have you tested with the latest master to confirm the issue still exists?
• Have you searched for related issues/PRs?
• What's the actual output vs expected output?
• [Optional] Sponsorship to speed up the bug fix or feature request (example)

Description

Null fields are dropped from HTTP request bodies. This is especially problematic for HTTP PATCH and PUT requests, where null values are semantically important.

Imagine you have a movie logging service, used to log movies you watch. You log having watched Blade and The Matrix, but then decide to clear their ratings so you can mull over your ratings a bit more. Here's how you might accomplish that with HTTP PUT and PATCH:

method  path    target body                          actual body
PUT     /log/1  {"name": "Blade", "rating": 5}       {"name": "Blade", "rating": 5}
PUT     /log/1  {"name": "Blade", "rating": null}    {"name": "Blade"}
PUT     /log/2  {"name": "The Matrix", "rating": 3}  {"name": "The Matrix", "rating": 3}
PATCH   /log/2  {"rating": null}                     {}

The problem, as shown by the "actual body" column above, is that openapi-generator-cli v7.6.0 python bindings drop null fields from requests. Therefore, it's impossible to make HTTP PUT and PATCH requests which explicitly clear fields. The root cause is this line in the python API client:

obj_dict = obj.model_dump(by_alias=True, exclude_unset=True)

Per BaseModel.model_dump(), the exclude_none controls the following behavior:

Whether to exclude fields that have a value of None.

openapi-generator version

This issue is present in "python" bindings generated by openapi-generator-cli v7.6.0 and latest. Earlier versions of openapi-generator-cli shipped a very different python code generator.

OpenAPI declaration file content or url

https://git.sr.ht/~jaudet/impedimenta/tree/main/item/openapi/sample-project/openapi/spec.yml

Generation Details

To generate API bindings:

git clone https://git.sr.ht/~jaudet/impedimenta
cd impedimenta/openapi/sample-project
./openapi/gen-bindings.sh

Steps to reproduce

To reproduce the issue, execute the above, then:

python3 -m unittest sample_project.tests

Several unit tests demonstrating the behavior will fail.

Related issues/PRs

No.

Suggest a fix

There are several possible solutions. One might be to apply this diff:

diff --git a/modules/openapi-generator/src/main/resources/python/api_client.mustache b/modules/openapi-generator/src/main/resources/python/api_client.mustache
index c7d4a6d700f..61d91641dc5 100644
--- a/modules/openapi-generator/src/main/resources/python/api_client.mustache
+++ b/modules/openapi-generator/src/main/resources/python/api_client.mustache
@@ -371,19 +371,12 @@ class ApiClient:
             )
         elif isinstance(obj, (datetime.datetime, datetime.date)):
             return obj.isoformat()
-
         elif isinstance(obj, dict):
             obj_dict = obj
+        elif hasattr(obj, "model_dump") and callable(obj.model_dump):
+            obj_dict = obj.model_dump(by_alias=True, exclude_unset=True)
         else:
-            # Convert model obj to dict except
-            # attributes `openapi_types`, `attribute_map`
-            # and attributes which value is not None.
-            # Convert attribute name to json key in
-            # model definition for request.
-            if hasattr(obj, 'to_dict') and callable(getattr(obj, 'to_dict')):
-                obj_dict = obj.to_dict()
-            else:
-                obj_dict = obj.__dict__
+            raise TypeError(f"Can't serialize obj: {obj}")
 
         return {
             key: self.sanitize_for_serialization(val)

As an aside, there's quite a bit of serialization logic in the python API client and models. This seems odd, given that pydantic models include logic for serialization via model_dump() and similar.

Start with modules/openapi-generator/src/main/resources/python/api_client.mustache and reproduce the issue using the linked sample project and its gen-bindings.sh script. Run python3 -m unittest sample_project.tests and verify that generated Python clients preserve explicitly null fields in PUT and PATCH request bodies while the existing serialization tests pass.