Romanitho / Romanitho/WingetIntunePackager

Icon lookup fails silently for all apps due to Google Image Search blocking automated requests

Open
#40 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
PowerShell
Stars
92
Forks
23
PR merge metrics
No merged PRs in 30d

Description

The current implementation fetches app icons by scraping Google Image Search (google.com/search?tbm=isch), which blocks automated/non-browser requests and returns no .Images array. This causes a null reference error on every app lookup, crashing the icon download step and leaving the icon field empty.
The fix replaces the single Google scraping call with a 4-source fallback chain:

  1. Winget manifest – checks the locale.en-US.yaml for a declared icon URL
  2. Clearbit Logo API – resolves a logo via the app's domain
  3. Google Favicon S2 API – fetches the publisher favicon (official API, no blocking)
  4. DuckDuckGo Image Search – last resort scraping fallback

The domain used for sources 2 and 3 is derived in priority order from PackageUrl (if present in the manifest), then the app name, then the publisher name — ensuring broad coverage even for packages with minimal manifest metadata.

I replaced lines 520 -524 with this code (AI generated):

#Get App icon - tries 4 sources in order, falls back gracefully
    $AppInfo.Icon = "$Location\$($AppInfo.ID).jpg"
    $IconUrl = $null

    # Domain ermitteln - Priorität: 1) PackageUrl, 2) App-Name, 3) Publisher-Name
    $PublisherDomain = $null
    if ($AppInfo.PackageUrl) {
        try { $PublisherDomain = ([System.Uri]$AppInfo.PackageUrl.Trim()).Host -replace '^www\.', '' } catch { }
    }
    if (!$PublisherDomain -and $AppInfo.PackageName) {
        $AppDomainGuess = ($AppInfo.PackageName -replace '[^a-zA-Z0-9]', '').ToLower()
        if ($AppDomainGuess.Length -ge 3) {
            $PublisherDomain = "$AppDomainGuess.com"
        }
    }
    if (!$PublisherDomain -and $AppInfo.Publisher) {
        $DomainGuess = ($AppInfo.Publisher -replace '\s+(LLC|Inc|Ltd|GmbH|Corp|Co|AG|BV|SAS|SRL|AB)\.?$', '' -replace '[^a-zA-Z0-9]', '').ToLower()
        if ($DomainGuess.Length -ge 3) {
            $PublisherDomain = "$DomainGuess.com"
        }
    }

    # Source 1: Winget-pkgs GitHub locale manifest (IconUrl field, no blocking)
    if (!$IconUrl) {
        try {
            $IconManifestUrl = "https://raw.githubusercontent.com/microsoft/winget-pkgs/master/manifests/$($AppFirstChar)/$($AppUrl)/$($AppVersion)/$($AppInfo.ID).locale.en-US.yaml"
            $LocaleManifest = (Invoke-WebRequest -UseBasicParsing -Uri $IconManifestUrl -ErrorAction Stop).Content
            $IconLineMatch = $LocaleManifest | Select-String -Pattern "(?m)^\s*-\s*Url:\s*(https?://\S+\.(?:png|jpg|jpeg|ico|svg))" | Select-Object -First 1
            if ($IconLineMatch) {
                $IconUrl = $IconLineMatch.Matches.Groups[1].Value.Trim()
                Write-Host "Icon source: Winget Manifest"
            }
        }
        catch { }
    }

    # Source 2: Clearbit Logo API
    if (!$IconUrl -and $PublisherDomain) {
        try {
            $ClearbitUrl = "https://logo.clearbit.com/$PublisherDomain"
            $TestResponse = Invoke-WebRequest -UseBasicParsing -Uri $ClearbitUrl -ErrorAction Stop
            if ($TestResponse.StatusCode -eq 200 -and $TestResponse.RawContentLength -gt 1000) {
                $IconUrl = $ClearbitUrl
                $AppInfo.Icon = "$Location\$($AppInfo.ID).png"
                Write-Host "Icon source: Clearbit"
            }
        }
        catch { }
    }

    # Source 3: Google Favicon S2 API
    if (!$IconUrl -and $PublisherDomain) {
        try {
            $FaviconUrl = "https://www.google.com/s2/favicons?domain=$PublisherDomain&sz=128"
            $FaviconResponse = Invoke-WebRequest -UseBasicParsing -Uri $FaviconUrl -ErrorAction Stop
            if ($FaviconResponse.StatusCode -eq 200 -and $FaviconResponse.RawContentLength -gt 500) {
                $IconUrl = $FaviconUrl
                $AppInfo.Icon = "$Location\$($AppInfo.ID).png"
                Write-Host "Icon source: Google Favicon S2"
            }
        }
        catch { }
    }

    # Source 4: DuckDuckGo Image Search (last resort)
    if (!$IconUrl) {
        try {
            $SearchQuery = [Uri]::EscapeDataString("$($AppInfo.PackageName) software logo")
            $DDGUrl = "https://duckduckgo.com/?q=$SearchQuery&iax=images&ia=images"
            $WebResponse = Invoke-WebRequest -Uri $DDGUrl -UseBasicParsing -ErrorAction Stop
            $ImgMatches = [regex]::Matches($WebResponse.Content, '"thumbnail":"(https?://[^"]+)"')
            if ($ImgMatches.Count -gt 1) {
                $IconUrl = $ImgMatches[1].Groups[1].Value
                Write-Host "Icon source: DuckDuckGo Image Search"
            }
        }
        catch { }
    }

    # Download icon, validate min. 1KB
    if ($IconUrl) {
        try {
            Invoke-WebRequest -Uri $IconUrl -OutFile $AppInfo.Icon -UseBasicParsing -ErrorAction Stop
            if ((Get-Item $AppInfo.Icon -ErrorAction SilentlyContinue).Length -lt 1024) {
                Remove-Item $AppInfo.Icon -Force -ErrorAction SilentlyContinue
                $AppInfo.Icon = $null
            }
        }
        catch {
            $AppInfo.Icon = $null
        }
    }
    else {
        $AppInfo.Icon = $null
    }

This fixed the issue for me

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the icon-download implementation around lines 520-524 and reproduce an app lookup that currently returns no image data and leaves the icon empty. Review the proposed Winget manifest, Clearbit, Google Favicon, and DuckDuckGo fallback sources, then verify that an app lookup no longer crashes and that a valid icon is downloaded or the icon is cleared gracefully when all sources fail.

Written by the indexing model from the issue text.

Assessment

Tech stack
powershell
Domain
tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.