Error in pydantic when running python gradio_chat.py --baseonly

Following local finetuning README

Ran python gradio_chat.py --baseonly

Got:

(phi-3-env) hayden@XPS15:/mnt/d/phi-3-env/inference$ python gradio_chat.py --baseonly
Number of GPUs available: 1
Running on device: cuda
CPU threads: 10
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
`flash-attention` package not found, consider installing for better performance: No module named 'flash_attn'.
Current `flash-attention` does not support `window_size`. Either upgrade or use `attn_implementation='eager'`.
Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [05:08<00:00, 154.26s/it]
Number of GPUs available: 1
Model ../model-cache/microsoft/Phi-3-mini-4k-instruct loaded successfully on cuda
INFO:     Started server process [17975]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:7860 (Press CTRL+C to quit)
INFO:     127.0.0.1:59874 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:59874 - "GET /assets/index-D6iiusuW.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:59874 - "GET /assets/index-Ds_LdHYW.css HTTP/1.1" 200 OK
INFO:     127.0.0.1:59874 - "GET /assets/svelte/svelte.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:59874 - "GET /assets/Index-DvJ399W-.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:59874 - "GET /assets/Index-B4pUrfBk.css HTTP/1.1" 200 OK
INFO:     127.0.0.1:59874 - "GET /assets/_commonjsHelpers-BosuxZz1.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:59874 - "GET /assets/index-Ds_LdHYW.css HTTP/1.1" 200 OK
INFO:     127.0.0.1:59874 - "GET /info HTTP/1.1" 200 OK
INFO:     127.0.0.1:59874 - "GET /theme.css HTTP/1.1" 200 OK
INFO:     127.0.0.1:59874 - "GET /assets/Button-CTZL5Nos.css HTTP/1.1" 200 OK
INFO:     127.0.0.1:59874 - "GET /assets/Blocks-xuLVTZz4.css HTTP/1.1" 200 OK
INFO:     127.0.0.1:59874 - "GET /assets/Blocks-BrGSw8d-.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:59874 - "GET /assets/Button-BM3Gfoxv.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:59888 - "GET /assets/Index-COkUHsKJ.css HTTP/1.1" 200 OK
INFO:     127.0.0.1:59874 - "GET /assets/Index-DPQUg1KC.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:59888 - "GET /assets/Index-CptIZeFZ.css HTTP/1.1" 200 OK
INFO:     127.0.0.1:59874 - "GET /assets/Textbox-D8IAzrZj.css HTTP/1.1" 200 OK
INFO:     127.0.0.1:59888 - "GET /assets/Index-B0JJ6p9c.css HTTP/1.1" 200 OK
INFO:     127.0.0.1:59874 - "GET /assets/Example-Cj3ii62O.css HTTP/1.1" 200 OK
INFO:     127.0.0.1:59874 - "GET /assets/Index-INgKs-Lw.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:59888 - "GET /assets/Index-CJeiS5ET.css HTTP/1.1" 200 OK
INFO:     127.0.0.1:59874 - "GET /assets/Index-DPgNZtxV.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:59888 - "GET /assets/Textbox-OSHpBx5r.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:59874 - "GET /assets/BlockTitle-BG3S3JH7.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:59888 - "GET /assets/Info-Cs8uP4Sq.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:59874 - "GET /assets/Check-Ck0iADAu.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:59888 - "GET /assets/Copy-ZPOKSMtK.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:59874 - "GET /assets/Index-BEdjhdex.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:59888 - "GET /assets/Example-C7XUkkid.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:59888 - "GET /assets/Index-DDCF2BFd.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:59874 - "GET /assets/Index-D-Msc1SI.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:59874 - "GET /favicon.ico HTTP/1.1" 200 OK
INFO:     127.0.0.1:59898 - "POST /queue/join HTTP/1.1" 500 Internal Server Error
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/pydantic/type_adapter.py", line 277, in _init_core_attrs
    self._core_schema = _getattr_no_parents(self._type, '__pydantic_core_schema__')
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/pydantic/type_adapter.py", line 119, in _getattr_no_parents
    raise AttributeError(attribute)
AttributeError: __pydantic_core_schema__

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/uvicorn/protocols/http/h11_impl.py", line 406, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/uvicorn/middleware/proxy_headers.py", line 70, in __call__
    return await self.app(scope, receive, send)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/starlette/applications.py", line 113, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/starlette/middleware/errors.py", line 187, in __call__
    raise exc
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/starlette/middleware/errors.py", line 165, in __call__
    await self.app(scope, receive, _send)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/starlette/_exception_handler.py", line 62, in wrapped_app
    raise exc
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/starlette/_exception_handler.py", line 51, in wrapped_app
    await app(scope, receive, sender)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/starlette/routing.py", line 715, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/starlette/routing.py", line 735, in app
    await route.handle(scope, receive, send)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/starlette/routing.py", line 460, in handle
    await self.app(scope, receive, send)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/starlette/applications.py", line 113, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/starlette/middleware/errors.py", line 187, in __call__
    raise exc
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/starlette/middleware/errors.py", line 165, in __call__
    await self.app(scope, receive, _send)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/gradio/route_utils.py", line 713, in __call__
    await self.simple_response(scope, receive, send, request_headers=headers)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/gradio/route_utils.py", line 729, in simple_response
    await self.app(scope, receive, send)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/starlette/_exception_handler.py", line 62, in wrapped_app
    raise exc
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/starlette/_exception_handler.py", line 51, in wrapped_app
    await app(scope, receive, sender)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/starlette/routing.py", line 715, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/starlette/routing.py", line 735, in app
    await route.handle(scope, receive, send)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/starlette/routing.py", line 288, in handle
    await self.app(scope, receive, send)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/starlette/routing.py", line 76, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/starlette/_exception_handler.py", line 62, in wrapped_app
    raise exc
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/starlette/_exception_handler.py", line 51, in wrapped_app
    await app(scope, receive, sender)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/starlette/routing.py", line 73, in app
    response = await f(request)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/fastapi/routing.py", line 291, in app
    solved_result = await solve_dependencies(
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/fastapi/dependencies/utils.py", line 639, in solve_dependencies
    ) = await request_body_to_args(  # body_params checked above
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/fastapi/dependencies/utils.py", line 810, in request_body_to_args
    fields_to_extract = get_model_fields(first_field.type_)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/fastapi/_compat.py", line 283, in get_model_fields
    return [
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/fastapi/_compat.py", line 284, in <listcomp>
    ModelField(field_info=field_info, name=name)
  File "<string>", line 6, in __init__
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/fastapi/_compat.py", line 109, in __post_init__
    self._type_adapter: TypeAdapter[Any] = TypeAdapter(
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/pydantic/type_adapter.py", line 264, in __init__
    self._init_core_attrs(rebuild_mocks=False)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/pydantic/type_adapter.py", line 142, in wrapped
    return func(self, *args, **kwargs)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/pydantic/type_adapter.py", line 284, in _init_core_attrs
    self._core_schema = _get_schema(self._type, config_wrapper, parent_depth=self._parent_depth)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/pydantic/type_adapter.py", line 102, in _get_schema
    schema = gen.generate_schema(type_)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 512, in generate_schema
    schema = self._generate_schema_inner(obj)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 768, in _generate_schema_inner
    return self._annotated_schema(obj)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 1822, in _annotated_schema
    schema = self._apply_annotations(source_type, annotations)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 1890, in _apply_annotations
    schema = get_inner_schema(source_type)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/pydantic/_internal/_schema_generation_shared.py", line 83, in __call__
    schema = self._handler(source_type)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 1972, in new_handler
    schema = metadata_get_schema(source, get_inner_schema)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 1968, in <lambda>
    lambda source, handler: handler(source)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/pydantic/_internal/_schema_generation_shared.py", line 83, in __call__
    schema = self._handler(source_type)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 1871, in inner_handler
    schema = self._generate_schema_inner(obj)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 789, in _generate_schema_inner
    return self.match_type(obj)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 871, in match_type
    return self._match_generic_type(obj, origin)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 895, in _match_generic_type
    return self._union_schema(obj)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 1207, in _union_schema
    choices.append(self.generate_schema(arg))
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 512, in generate_schema
    schema = self._generate_schema_inner(obj)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 789, in _generate_schema_inner
    return self.match_type(obj)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 875, in match_type
    return self._unknown_type_schema(obj)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 415, in _unknown_type_schema
    raise PydanticSchemaGenerationError(
pydantic.errors.PydanticSchemaGenerationError: Unable to generate pydantic-core schema for <class 'starlette.requests.Request'>. Set `arbitrary_types_allowed=True` in the model_config to ignore this error or implement `__get_pydantic_core_schema__` on your type to fully support it.

If you got this error by calling handler(<some type>) within `__get_pydantic_core_schema__` then you likely need to call `handler.generate_schema(<some type>)` since we do not call `__get_pydantic_core_schema__` on `<some type>` otherwise to avoid infinite recursion.

For further information visit https://errors.pydantic.dev/2.8/u/schema-for-unknown-type

Start with the local finetuning README and gradio_chat.py, then reproduce python gradio_chat.py --baseonly and submit a request to the running interface. Trace the /queue/join failure through the reported FastAPI, Gradio, and Pydantic stack; done means the request succeeds without the TypeAdapter error.