[API] No reliable way to query for remote status of multiple packages
- Dominant language
- C#
- Stars
- 11.5k
- Forks
- 960
- PR merge metrics
- No merged PRs in 30d
Description
I have a list of packages, *e.g.* `"git vim vlc"`, and I would like to get their respective version information from the online repositories. Ideally this would be through the API, but I realise `choco` doesn’t seem to provide this either.
## Summary of issues
- `choco search` without arguments takes a very long time and downloads a lot of data
- `choco search` without arguments does not actually return all packages
- `--only-id` only works with a single string or substring
- `--page` does not appear to reduce the server load or transferred data amount
- `--id-starts-with` omits a lot of results
- `--page` does not appear to work with `--id-starts-with`
## Description of attempts
Using the command line I can do something like this in bash:
```
for pkg in git vim vlc; do choco search -r -e --by-id-only $pkg; done
```
With the following output:
```
git|2.28.0
vim|8.2.1651
vlc|3.0.11
```
The drawback is that this performs one query per package. If I have 50 packages this becomes cumbersome. With more packages it becomes unrealistic (my IP has been banned from chocolatey.org) and there doesn’t seem to be a middle ground between exact search and “list all packages” that suits my simple need.
I can search for all packages and filter manually using `grep`:
```
choco search -r | grep '^\(git\|vim\|vlc\)|'
```
But this query takes forever (70 seconds) and downloads about 90 MiB of data from the server. Also, plenty of packages are missing (as reported in #2001) and this appears to be non-deterministic.
Chunking the search results in pages (`choco search -r --page-size=100 --page=0`) to reduce the load on the server actually takes exactly as long; it’s as if paging was performed on the client side, making it useless for that purpose.
I noticed that `vlc` and `vim` both start with `v` so I thought I’d search for packages that start with `v` and filter the results, to reduce the work on the server:
```
choco search -r --id-starts-with v | grep '\<\(vim\|vlc\)\>'
````
But while `choco` indeed returns 21 packages starting with `v`, they neither include `vim` nor `vlc`. This may be related to #1836 and the fact that `--id-starts-with` causes `choco` to return significantly fewer packages than usual (in contrast, `choco search -r | grep '^v'` returns 365 packages). So I thought, maybe these packages are on the second result page:
```
choco search -r --id-starts-with v --page 1
```
But this returns zero results. So maybe I can increase the page size:
```
choco search -r --id-starts-with v --page-size 2000
```
But this still returns the same 21 packages as before.
## Feature requests
The help for `choco search` says the argument is a “filter” but does not specify what a filter is. If I understand correctly, it is just a verbatim string (or substring). I think such a syntax extension could be helpful:
```
choco search -r -e --by-id-only git vim vlc # multiple arguments for search
choco search -r -e --by-id-only git,vim,vlc # comma-separated filter
choco search -r -e --by-id-only 'git|vim|vlc' # regular expression
choco search -r -e --by-id-only '(git|vim|vlc)' # alternate regex syntax
```
Note that the Debian package manager allow the following convenient invocation: `apt search --names-only '^(git|vim|vlc)$'`
This may also be what #1345 is about, but the request is a bit unclear to me. There is also [mention of `--order-by-recent`](https://github.com/chocolatey/choco/issues/714#issuecomment-217931062) but I believe this hasn’t been implemented yet.
Contributor guide
Research direction
Start with the choco search entry point and its help text, then trace how --by-id-only, --id-starts-with, --page, and --page-size are handled. Define the intended multi-package query and verify that remote filtering and pagination reduce transferred data while returning all requested package versions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- api, cli
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100