hashicorp / hashicorp/vagrant_cloud
`vagrant cloud search` is slow due to `O(n+1)` API requests
- Dominant language
- Ruby
- Stars
- 16
- Forks
- 13
- PR merge metrics
- No merged PRs in 30d
Description
A simple call to `vagrant cloud search ubuntu2204` takes upwards of 15 seconds to run on high end hardware... adding `--debug-timestamp` I noticed it hangs on API requests and when I searched the debug output I noticed the following for one search:
```log
2022-10-26 00:39:36 [ INFO] excon REQUEST method=:get identifier="fb82e6ec-bf46-44de-b432-369cc1a6e8cd" url="https://vagrantcloud.com/api/v1/search" query="q: ubuntu2204" headers="Host: vagrantcloud.com:443, Accept: application/json, Content-Type: application/json, X-Request-Id: fb82e6ec-bf46-44de-b432-369cc1a6e8cd" duration="83ms"
2022-10-26 00:39:36 [ INFO] excon REQUEST method=:get identifier="9b6f83e7-f51c-45a8-b4b3-e0a23250937a" url="https://vagrantcloud.com/api/v1/user/generic" headers="Host: vagrantcloud.com:443, Accept: application/json, Content-Type: application/json, X-Request-Id: 9b6f83e7-f51c-45a8-b4b3-e0a23250937a" duration="40ms"
2022-10-26 00:39:40 [ INFO] excon REQUEST method=:get identifier="48e70e0b-83f1-4e9e-b445-ba9d786c43d1" url="https://vagrantcloud.com/api/v1/user/roboxes" headers="Host: vagrantcloud.com:443, Accept: application/json, Content-Type: application/json, X-Request-Id: 48e70e0b-83f1-4e9e-b445-ba9d786c43d1" duration="34ms"
2022-10-26 00:39:44 [ INFO] excon REQUEST method=:get identifier="7728c2dc-9f5f-4df0-8dcf-6e95a02aeaa2" url="https://vagrantcloud.com/api/v1/user/fasmat" headers="Host: vagrantcloud.com:443, Accept: application/json, Content-Type: application/json, X-Request-Id: 7728c2dc-9f5f-4df0-8dcf-6e95a02aeaa2" duration="34ms"
2022-10-26 00:39:44 [ INFO] excon REQUEST method=:get identifier="212100db-e3b4-4240-96c4-19c556e59cca" url="https://vagrantcloud.com/api/v1/user/techchad2022" headers="Host: vagrantcloud.com:443, Accept: application/json, Content-Type: application/json, X-Request-Id: 212100db-e3b4-4240-96c4-19c556e59cca" duration="39ms"
2022-10-26 00:39:45 [ INFO] excon REQUEST method=:get identifier="b87e3ea7-5134-4b9b-8f50-e7b502522a9a" url="https://vagrantcloud.com/api/v1/user/ilionx" headers="Host: vagrantcloud.com:443, Accept: application/json, Content-Type: application/json, X-Request-Id: b87e3ea7-5134-4b9b-8f50-e7b502522a9a" duration="41ms"
2022-10-26 00:39:46 [ INFO] excon REQUEST method=:get identifier="61896816-1168-4462-b217-9a1b3e2d3c69" url="https://vagrantcloud.com/api/v1/user/jaredeh" headers="Host: vagrantcloud.com:443, Accept: application/json, Content-Type: application/json, X-Request-Id: 61896816-1168-4462-b217-9a1b3e2d3c69" duration="42ms"
2022-10-26 00:39:46 [ INFO] excon REQUEST method=:get identifier="34620a6d-acb7-4ae1-8184-0a88d79b2db4" url="https://vagrantcloud.com/api/v1/user/krec" headers="Host: vagrantcloud.com:443, Accept: application/json, Content-Type: application/json, X-Request-Id: 34620a6d-acb7-4ae1-8184-0a88d79b2db4" duration="43ms"
2022-10-26 00:39:47 [ INFO] excon REQUEST method=:get identifier="540a21a4-7ae4-48fd-8749-8a24c742dd0b" url="https://vagrantcloud.com/api/v1/user/marcinbojko" headers="Host: vagrantcloud.com:443, Accept: application/json, Content-Type: application/json, X-Request-Id: 540a21a4-7ae4-48fd-8749-8a24c742dd0b" duration="41ms"
2022-10-26 00:39:47 [ INFO] excon REQUEST method=:get identifier="5df81256-6c75-4bb2-be84-135886224fea" url="https://vagrantcloud.com/api/v1/user/maroskukan" headers="Host: vagrantcloud.com:443, Accept: application/json, Content-Type: application/json, X-Request-Id: 5df81256-6c75-4bb2-be84-135886224fea" duration="38ms"
```
Search results:
```log
2022-10-26 00:39:48 [ INFO] info: | NAME | VERSION | DOWNLOADS | PROVIDERS |
2022-10-26 00:39:48 [ INFO] info: +----------------------------+----------------+-----------+--------------------------------------------------------+
2022-10-26 00:39:48 [ INFO] info: | generic/ubuntu2204 | 4.1.18 | 26,057 | vmware_desktop, virtualbox, parallels, libvirt, hyperv |
2022-10-26 00:39:48 [ INFO] info: | roboxes/ubuntu2204 | 4.1.18 | 1,891 | virtualbox, parallels, vmware_desktop, hyperv, libvirt |
2022-10-26 00:39:48 [ INFO] info: | fasmat/ubuntu2204-desktop | 22.0509.1 | 334 | virtualbox, vmware_desktop |
2022-10-26 00:39:48 [ INFO] info: | techchad2022/ubuntu2204 | 0.2 | 187 | virtualbox, libvirt |
2022-10-26 00:39:48 [ INFO] info: | ilionx/ubuntu2204-minikube | 1.1.0-20220825 | 84 | virtualbox |
2022-10-26 00:39:48 [ INFO] info: | techchad2022/lamp | 0.2 | 74 | virtualbox, libvirt |
2022-10-26 00:39:48 [ INFO] info: | jaredeh/ubuntu2204-server | 0.1.0 | 71 | libvirt |
2022-10-26 00:39:48 [ INFO] info: | krec/ubuntu2204-german | 20220904.0.1 | 71 | virtualbox |
2022-10-26 00:39:48 [ INFO] info: | marcinbojko/ubuntu2204 | 1.0.0 | 53 | virtualbox |
2022-10-26 00:39:48 [ INFO] info: | maroskukan/ubuntu2204 | 2022.07.26 | 36 | virtualbox, hyperv |
2022-10-26 00:39:48 [ INFO] info: +----------------------------+----------------+-----------+--------------------------------------------------------+
```
It appears that the search is performing a lookup on each user/org in the result set - thus `O(n+1)` complexity. I checked the timestamps and 12 seconds elapses largely due to connection overhead.
I don't see anything in the output that would necessitate the user lookup... I assume that's for some other consumer of these APIs? Am I missing something that requires the extra lookups? Perhaps a workaround for stale data in the initial query response?
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.