PowerShell / PowerShell/PSResourceGet

Speed up NuGet v3 queries by caching and/or ETag

Open
#1,603 3 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Area-Find Issue-Performance
Dominant language
C#
Stars
576
Forks
114
Avg merge
1d 2h
Merged PRs (30d)
7

Description

Edit: I assumed wrong in my OP, but there are other things that can be done to speed up NuGet v3 queries described in following comment:

Prerequisites
  • Write a descriptive title.
  • Make sure you are able to repro it on the latest released version
  • Search the existing issues.
Steps to reproduce

I'm experimenting with pwsh.gallery + PSResourceGet.

Added it as repository like so:

Register-PSResourceRepository -Name 'pwsh.gallery' -Uri 'https://pwsh.gallery/index.json'

Then I wanted to see how fast it can return latest version of a given module. I did:

Find-PSResource -Repository 'pwsh.gallery' -Name 'Az.Accounts' -Verbose -Debug

I notice that PSResourceGet does three API calls where the first and last seems totally unneccessary for getting latest version.

One can determine latest version by just getting https://pwsh.gallery/az.accounts/index.json, or one could go straight to https://pwsh.gallery/az.accounts/page/latest.json.

PS > Find-PSResource -Repository 'pwsh.gallery' -Name 'Az.Accounts' -Verbose -Debug
DEBUG: In FindPSResource::ProcessResourceNameParameterSet()
DEBUG: Filtering package name(s) on wildcards
DEBUG: In FindHelper::FindByResourceName()
DEBUG: Parameters passed in >>> Name: 'Az.Accounts'; ResourceType: 'None'; VersionRange: ''; NuGetVersion: ''; VersionType: 'NoVersion'; Version: ''; Prerelease: 'False'; Tag: ''; Repository: 'pwsh.gallery'; IncludeDependencies 'False'
DEBUG: Searching through repository 'pwsh.gallery'
DEBUG: In FindHelper::SearchByNames()
DEBUG: No version specified, package name is specified
DEBUG: In V3ServerAPICalls::FindName()
DEBUG: In V3ServerAPICalls::FindNameHelper()
DEBUG: In V3ServerAPICalls::GetVersionedPackageEntriesFromRegistrationsResource()
DEBUG: In V3ServerAPICalls::GetResourcesFromServiceIndex()
DEBUG: In V3ServerAPICalls::HttpRequestCall()
DEBUG: Request url is 'https://pwsh.gallery/index.json'
DEBUG: In V3ServerAPICalls::FindRegistrationsBaseUrl()
DEBUG: In V3ServerAPICalls::GetVersionedResponsesFromRegistrationsResource()
DEBUG: In V3ServerAPICalls::HttpRequestCall()
DEBUG: Request url is 'https://pwsh.gallery/az.accounts/index.json'
DEBUG: In V3ServerAPICalls::GetMetadataElementsFromResponse()
DEBUG: In V3ServerAPICalls::GetMetadataElementFromItemsElement()
DEBUG: In V3ServerAPICalls::GetMetadataElementFromIdLinkElement()
DEBUG: In V3ServerAPICalls::HttpRequestCall()
DEBUG: Request url is 'https://pwsh.gallery/az.accounts/page/recent.json'
DEBUG: In V3ServerAPICalls::IsLatestVersionFirstForSearch()
DEBUG: 'Az.Accounts' version parsed as '2.16.0'
DEBUG: Found package 'Az.Accounts' version '2.16.0'
DEBUG: Package 'Az.Accounts' returned from server
DEBUG: Package 'Az.Accounts' was previously discovered and returned

Name        Version Prerelease Repository   Description
----        ------- ---------- ----------   -----------
Az.Accounts 2.16.0             pwsh.gallery

PS >

Questions:

  1. Wouldn't it be better to just get <nugetv3api>/az.accounts/index.json or <nugetv3api>/az.accounts/page/latest.json?
  2. Would it make sense to support -Version 'Latest' as input to Find-PSResource?
Expected behavior

When getting latest version only, Find-PSResource does just enough API calls, to 1) speed up end user experience, and 2) not put unneccessary load on the target repository/API.

Actual behavior

Find-PSResource gets <nugetv3api>/az.accounts/page/recent.json when getting latest version only.

Error details

No response

Environment data
PowerShell v7.4.1 x64 on Windows 11 23H2
Microsoft.PowerShell.PSResourceGet v1.0.3
Visuals

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the Find-PSResource path shown in the debug output, especially V3ServerAPICalls::FindName and its registration-resource HTTP calls. Reproduce the Az.Accounts query against the pwsh.gallery example, then inspect the linked comment for the revised optimization scope. Done means latest-version queries avoid unnecessary NuGet v3 requests while returning the expected result.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp, powershell
Domain
cli, tooling
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.