typesense / typesense/typesense
Adding nested fields with type `auto` causes document indexing errors
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 26.6k
- Forks
- 973
- Avg merge
- 18h 45m
- Merged PRs (30d)
- 4
Description
Description
I've encountered an issue when adding nested fields of type auto. Creating all the nested type auto fields when the collection is created works, but creating one in a separate operation seems to cause field type errors. I'm not sure if I'm missing something or if this is a bug.
Steps to reproduce
-
Create a collection with a nested field of type
autoe.g:curl "https://****.typesense.net/collections" \ -X POST \ -H "Content-Type: application/json" \ -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" \ -d '{ "enable_nested_fields": true, "fields": [ { "name": "name", "type": "string" }, { "facet": true, "name": "nested.foo", "optional": true, "type": "auto" } ], "name": "test_collection" }'Response:
{ "created_at": 1724205875, "default_sorting_field": "", "enable_nested_fields": true, "fields": [ { "facet": false, "index": true, "infix": false, "locale": "", "name": "name", "optional": false, "sort": false, "stem": false, "store": true, "type": "string" }, { "facet": true, "index": true, "infix": false, "locale": "", "name": "nested.foo", "optional": true, "sort": false, "stem": false, "store": true, "type": "auto" } ], "name": "test_collection", "num_documents": 0, "symbols_to_index": [], "token_separators": [] } -
Upload a document that adheres to the schema:
curl "https://****.typesense.net/collections/test_collection/documents" \ -X POST \ -H "Content-Type: application/json" \ -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" \ -d '{ "name": "mydoc", "nested": { "foo": "hello" } }'This is successfully inserted:
{ "id": "0", "name": "mydoc", "nested": { "foo": "hello" } } -
Update the collection with another "sibling" nested field of type
auto:curl "https://****.typesense.net/collections/test_collection" \ -X PATCH \ -H "Content-Type: application/json" \ -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" \ -d '{ "fields": [ { "name": "nested.bar", "type": "auto", "optional": true, "facet": true } ] }'Response:
{ "fields": [ { "facet": true, "name": "nested.bar", "optional": true, "type": "auto" } ] } -
Upload another document that adheres to the new schema:
curl "https://****.typesense.net/collections/test_collection/documents" \ -X POST \ -H "Content-Type: application/json" \ -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" \ -d '{ "name": "indeedadoc", "nested": { "foo": "hello", "bar": "world" } }'The document fails to index with an error about
nested.bar's type:{ "message": "Field `nested.bar` has an incorrect type." }
This does work as expected if I create both nested fields at once, however:
-
Create the collection with both nested fields in the schema:
curl "https://****.typesense.net/collections" \ -X POST \ -H "Content-Type: application/json" \ -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" \ -d '{ "enable_nested_fields": true, "fields": [ { "name": "name", "type": "string" }, { "facet": true, "name": "nested.foo", "optional": true, "type": "auto" }, { "facet": true, "name": "nested.bar", "optional": true, "type": "auto" } ], "name": "test_collection2" }'Result:
{ "created_at": 1724205997, "default_sorting_field": "", "enable_nested_fields": true, "fields": [ { "facet": false, "index": true, "infix": false, "locale": "", "name": "name", "optional": false, "sort": false, "stem": false, "store": true, "type": "string" }, { "facet": true, "index": true, "infix": false, "locale": "", "name": "nested.foo", "optional": true, "sort": false, "stem": false, "store": true, "type": "auto" }, { "facet": true, "index": true, "infix": false, "locale": "", "name": "nested.bar", "optional": true, "sort": false, "stem": false, "store": true, "type": "auto" } ], "name": "test_collection2", "num_documents": 0, "symbols_to_index": [], "token_separators": [] } -
Insert a document that conforms to the schema:
curl "https://****.typesense.net/collections/test_collection2/documents" \ -X POST \ -H "Content-Type: application/json" \ -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" \ -d '{ "name": "indeedadoc", "nested": { "foo": "hello", "bar": "world" } }'The document inserts successfully:
{ "id": "0", "name": "indeedadoc", "nested": { "bar": "world", "foo": "hello" } }
The two collection schemas are identical when fetched, other than name and created_at:
curl "https://****.typesense.net/collections/test_collection" \
-X GET \
-H "Content-Type: application/json" \
-H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}"
Result:
{
"created_at": 1724205875,
"default_sorting_field": "",
"enable_nested_fields": true,
"fields": [
{
"facet": false,
"index": true,
"infix": false,
"locale": "",
"name": "name",
"optional": false,
"sort": false,
"stem": false,
"store": true,
"type": "string"
},
{
"facet": true,
"index": true,
"infix": false,
"locale": "",
"name": "nested.foo",
"optional": true,
"sort": false,
"stem": false,
"store": true,
"type": "auto"
},
{
"facet": true,
"index": true,
"infix": false,
"locale": "",
"name": "nested.bar",
"optional": true,
"sort": false,
"stem": false,
"store": true,
"type": "auto"
}
],
"name": "test_collection",
"num_documents": 1,
"symbols_to_index": [],
"token_separators": []
}
curl "https://****.typesense.net/collections/test_collection2" \
-X GET \
-H "Content-Type: application/json" \
-H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}"
Result:
{
"created_at": 1724205997,
"default_sorting_field": "",
"enable_nested_fields": true,
"fields": [
{
"facet": false,
"index": true,
"infix": false,
"locale": "",
"name": "name",
"optional": false,
"sort": false,
"stem": false,
"store": true,
"type": "string"
},
{
"facet": true,
"index": true,
"infix": false,
"locale": "",
"name": "nested.foo",
"optional": true,
"sort": false,
"stem": false,
"store": true,
"type": "auto"
},
{
"facet": true,
"index": true,
"infix": false,
"locale": "",
"name": "nested.bar",
"optional": true,
"sort": false,
"stem": false,
"store": true,
"type": "auto"
}
],
"name": "test_collection2",
"num_documents": 1,
"symbols_to_index": [],
"token_separators": []
}
Expected Behavior
Indexing a document should work the same way when a nested field with type auto is either:
- Present when the collection is initially created
- Added after the collection is created
Actual Behavior
Adding a document fails when an nested auto field is added after a collection is created.
Metadata
Typesense Version: v27.0.rc34
OS: Typesense Cloud
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Run the curl reproduction in the issue, comparing a nested auto field declared at collection creation with one added through the collection PATCH endpoint. Trace the nested-field schema and document-indexing paths for the mismatch. Done means documents containing the newly added nested auto field index successfully, matching the behavior of fields declared initially.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- backend, search
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 42/100