Research improving jsonschema validation performance
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 6.5k
- Forks
- 787
- PR merge metrics
- No merged PRs in 30d
Description
Right now, JSON schema validation is quite slow.
This is primarily noticable when registering a pack with a lot of resources / actions.
To put things into context - with my action performance registration improvements from https://github.com/StackStorm/st2/pull/4846#issuecomment-787050508, it now takes around 40 seconds to register all the actions from the aws pack. Out of those 40 seconds, 15-20 are spent on JSON schema validation.
We should look into options such as https://horejsek.github.io/python-fastjsonschema/.
Right now we utilize / rely on some of the jsonschema library internal API which means that the change likely won't be just drop in and replace and will require some (perhaps non-trivial amount of) work.
See https://github.com/StackStorm/st2/pull/4846#issuecomment-787050508 for some more context and details.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Review the action registration performance context in pull request 4846 and trace the current JSON schema validation usage, including its reliance on jsonschema internals. Compare possible validation approaches such as fastjsonschema and benchmark registration of the aws pack. Done means validation preserves existing behavior while substantially reducing registration time.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- performance
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100