DamianEdwards / DamianEdwards/kusto-cli
Stream query file contents to kusto
Nobody has claimed this yet.
- Dominant language
- C#
- Stars
- 8
- Forks
- 3
- Avg merge
- 1d 35m
- Merged PRs (30d)
- 12
Description
Instead of reading the specified --file into memory as a string to assign it to the kusto HTTP request body, stream it directly from the file to the request body in order to reduce memory use. Might be better to only do this for files above a certain size (e.g. 4KB?) but also might be easier to just do it all the time. The goal here is to reduce memory use and improve performance of the CLI when executing a large query.
This might impact the Kusto query parsing and syntax validation we do before sending the request though so that needs some thought, e.g. can we do the checking in chunks, should we support disabling the checking or opting into streaming mode (which would disable syntax checking), or support disabling the syntax checking, etc.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Trace the CLI's --file handling through construction of the Kusto HTTP request, starting where the file is read into a string. Determine how streaming affects Kusto query parsing and syntax validation, then define behavior for large files and validation before measuring memory use with a large query. Done means file contents stream to the request without the current full in-memory string and the validation behavior is explicit.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, csharp
- Domain
- api, cli, cloud
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100