Possible schema hashing false mismatch due to unstable msgpack map serialization
Nobody has claimed this yet.
- Dominant language
- Lua
- Stars
- 43
- Forks
- 17
- Avg merge
- 28m
- Merged PRs (30d)
- 2
Description
While working on the task, it was discovered that msgpack does not guarantee the order of elements when serializing map.
Input for example:
{
indexes = { {
parts = { {
exclude_null = false,
fieldno = 2
} }
},
[0] = {
parts = { {
exclude_null = false,
fieldno = 1
} }
}
}
}
Serializing output on different hosts:
��indexes���parts���exclude_null§fieldno��parts���exclude_null§fieldno
��indexes���parts���fieldno�exclude_null���parts���fieldno�exclude_null�
It can be seen that the fields exclude_null and fieldno have different order on different hosts.
Since this is used to get the hash and determine if the schema needs to be reloaded, we may get a false schema mismatch problem:
It should also be noted that a similar calculation approach is used in other places:
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the schema hashing logic at crud/common/schema.lua#L139, then compare the similar calculation in crud/common/sharding/utils.lua#L31. Reproduce the differing msgpack map order shown in the issue and verify that equivalent schemas produce the same hash across hosts without unnecessary reloads.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- lua
- Domain
- databases, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100