False positive cache hit

Overview

• Expected behavior: Observe a cache miss.
• Observed behavior: actions/cache reports a cache hit when no cache is present.

We are using actions/cache lookup only to check whether we have previously executed a job or not. It has been working OK until we identified two instances (there may be more) in which actions/cache reported a hit when it was supposed to be a miss. Here are a couple of examples:

Example 1

Originally posted here regarding this run:

This is what I see from calling gh cache ls --json id,key:

[END)
  {
    "id": 396,
    "key": "docker-dependencies-Linux-6d19bd7b980701f22c2af8652e06f82ed05ebeefd65b4033dc89fe1c3abecd38-build-main-pool"
  },
  {
    "id": 410,
    "key": "Ubuntu 22.04.5 LTS-R version 4.4.2 (2024-10-31)-1-7f0445647fb5a2aac10e6a6845e489bbe0cff9662f88cc3cfca939c2fd8e1b7f"
  },
  {
    "id": 412,
    "key": "Ubuntu 22.04.5 LTS-R version 4.4.2 (2024-10-31)-1-84b70f73dd13fdbaa431ccc8dcd245057774fde673c98049e605e1f0ede96df8"
  },
  {
    "id": 411,
    "key": "Ubuntu 22.04.5 LTS-R version 4.4.2 (2024-10-31)-1-460bc19305d5154a0091cdf9ecf08b62e3e63297ae0fbad8ddc190de03304291"
  },
  {
    "id": 408,
    "key": "Ubuntu 22.04.5 LTS-R version 4.4.2 (2024-10-31)-1-84b70f73dd13fdbaa431ccc8dcd245057774fde673c98049e605e1f0ede96df8"
  },
  {
    "id": 407,
    "key": "Ubuntu 22.04.5 LTS-R version 4.4.2 (2024-10-31)-1-7f0445647fb5a2aac10e6a6845e489bbe0cff9662f88cc3cfca939c2fd8e1b7f"
  },
  {
    "id": 409,
    "key": "Ubuntu 22.04.5 LTS-R version 4.4.2 (2024-10-31)-1-460bc19305d5154a0091cdf9ecf08b62e3e63297ae0fbad8ddc190de03304291"
  },
  {
    "id": 393,
    "key": "Ubuntu 22.04.5 LTS-R version 4.4.2 (2024-10-31)-1-4c07a1f8fdd193a63c4c9ec0d23873a8a5822196c33e385f05398cd9b830e0aa"
  },
  {
    "id": 406,
    "key": "docker-dependencies-Linux-6d19bd7b980701f22c2af8652e06f82ed05ebeefd65b4033dc89fe1c3abecd38-clarify-impure"
  },
  {
    "id": 397,
    "key": "docker-dependencies-Linux-6d19bd7b980701f22c2af8652e06f82ed05ebeefd65b4033dc89fe1c3abecd38-128-remove-old-action"
  }
]

And this is what I see from the step of that run:

Run actions/cache@v4
  with:
    key: docker-dependencies-Linux-f4bcfb55392f2a994483280942820d4c4affe91f35b437367ba7e4c0b6021444-51-check-gi-density
    lookup-only: true
    path: ./DESCRIPTION
    enableCrossOsArchive: false
    fail-on-cache-miss: false
    save-always: false
  env:
    REGISTRY: cfaprdbatchcr.azurecr.io
    IMAGE_NAME: cfa-epinow2-pipeline
Lookup only - skipping download
Cache found and can be restored from key: docker-dependencies-Linux-f4bcfb55392f2a994483280942820d4c4affe91f35b437367ba7e4c0b6021444-51-check-gi-density

So I have no idea why it's making a hit since there's no hash with f4bc* in the list of existing caches!

Example 2:

Originally posted here. Features this run:

Thanks for the example, @dylanhmorris! Here is the output I'm getting from checking the cache:

gh cache ls --json id,key

[
    {
        "id": 1215,
        "key": "Ubuntu 22.04.5 LTS-R version 4.4.2 (2024-10-31)-1-f85fa02bc57d3f407305e6d5bf93e59911ca7e0eb3a184f5c9f9518c5000d8af"
    },
    {
        "id": 1214,
        "key": "docker-dependencies-Linux-02b5e2d6aef8dd3a4ea8f51b074d3269bab5dfef8507c178d50fc75350e7aff1-dhm-run-pyrenew-h"
    },
    {
        "id": 1191,
        "key": "Windows Server 2022 x64 (build 20348)-R version 4.4.2 (2024-10-31 ucrt)-1-d47748fab30e34e26ffc10f8970c48c11008041b22e082e2ed8525c822f64f38"
    },
    {
        "id": 1192,
        "key": "Windows Server 2022 x64 (build 20348)-R version 4.4.0 (2024-04-24 ucrt)-1-2e815b11189127381077466a7f7b248e3f149541715601ebbe42fb3ef3b04309"
    },
    {
        "id": 1213,
        "key": "Ubuntu 22.04.5 LTS-R version 4.4.2 (2024-10-31)-1-3942e21f75d57e70130b5aba1366b5c20bad24e308e50eb7528285bb009c7895"
    },
    {
        "id": 1212,
        "key": "macOS Sonoma 14.7.2-R version 4.4.0 (2024-04-24)-1-3c831122a1d6cb7b3298ba9106e77bfd313c2a4953e41041c144b9abc49d59fb"
    },
    {
        "id": 1211,
        "key": "macOS Sonoma 14.7.2-R version 4.4.2 (2024-10-31)-1-552fe97f674812742e51a10283f3c4130b22593f0a62efba81d78ec07611d42d"
    },
    {
        "id": 1210,
        "key": "Ubuntu 22.04.5 LTS-R version 4.4.0 (2024-04-24)-1-75a8387d30a9e32c76d0e5883aedd4e6c6742016baa21d0c84e59568d8b2b50b"
    },
    {
        "id": 1189,
        "key": "macOS Sonoma 14.7.2-R version 4.4.0 (2024-04-24)-1-a46748d991d0f385a4b67206662ead3630e350fec232e19861a724661f8d32ea"
    },
    {
        "id": 1188,
        "key": "macOS Sonoma 14.7.2-R version 4.4.2 (2024-10-31)-1-bcd1e597b03282c6da48a8b4f3146cdb20d08a37f1a3d18462118d6f533a828f"
    },
    {
        "id": 1186,
        "key": "Ubuntu 22.04.5 LTS-R version 4.4.2 (2024-10-31)-1-cd166cd05a4988ff0f6212575fe8b681837fb8f5ff5ef5c03324bef0b65f6c36"
    },
    {
        "id": 1194,
        "key": "Ubuntu 22.04.5 LTS-R version 4.4.2 (2024-10-31)-1-17787be3eeb6cd3d84f97d7de16d975cbd461fd2a86065be16d1bbd18fa6f341"
    },
    {
        "id": 1187,
        "key": "Ubuntu 22.04.5 LTS-R version 4.4.0 (2024-04-24)-1-488e219a9a58ae5305b17158881bf08bba9ace8c082766f5f169607d7cd342eb"
    },
    {
        "id": 650,
        "key": "pre-commit-3|/opt/hostedtoolcache/Python/3.12.8/x64|7062add5de4c02de988411686969a115a1d328795dbff87a75840862398b679f"
    },
    {
        "id": 651,
        "key": "r-precommit-cache|/opt/hostedtoolcache/Python/3.12.8/x64|7062add5de4c02de988411686969a115a1d328795dbff87a75840862398b679f"
    },
    {
        "id": 1184,
        "key": "Linux-poetry"
    },
    {
        "id": 1209,
        "key": "Ubuntu 22.04.5 LTS-R version 4.4.2 (2024-10-31)-1-f85fa02bc57d3f407305e6d5bf93e59911ca7e0eb3a184f5c9f9518c5000d8af"
    },
    {
        "id": 1208,
        "key": "docker-dependencies-Linux-02b5e2d6aef8dd3a4ea8f51b074d3269bab5dfef8507c178d50fc75350e7aff1-dhm-pick-and-choose-denom"
    },
    {
        "id": 1150,
        "key": "Windows Server 2022 x64 (build 20348)-R version 4.4.0 (2024-04-24 ucrt)-1-2e815b11189127381077466a7f7b248e3f149541715601ebbe42fb3ef3b04309"
    },
    {
        "id": 1151,
        "key": "Windows Server 2022 x64 (build 20348)-R version 4.4.2 (2024-10-31 ucrt)-1-d47748fab30e34e26ffc10f8970c48c11008041b22e082e2ed8525c822f64f38"
    },
    {
        "id": 1207,
        "key": "macOS Sonoma 14.7.2-R version 4.4.0 (2024-04-24)-1-3c831122a1d6cb7b3298ba9106e77bfd313c2a4953e41041c144b9abc49d59fb"
    },
    {
        "id": 1162,
        "key": "Ubuntu 22.04.5 LTS-R version 4.4.2 (2024-10-31)-1-1f3f0940525e68bfdfd78468b47d5bdf086a8e7d193d498ec374308692ce80b2"
    },
    {
        "id": 1206,
        "key": "macOS Sonoma 14.7.2-R version 4.4.2 (2024-10-31)-1-552fe97f674812742e51a10283f3c4130b22593f0a62efba81d78ec07611d42d"
    },
    {
        "id": 1148,
        "key": "Ubuntu 22.04.5 LTS-R version 4.4.2 (2024-10-31)-1-cd166cd05a4988ff0f6212575fe8b681837fb8f5ff5ef5c03324bef0b65f6c36"
    },
    {
        "id": 1169,
        "key": "macOS Sonoma 14.7.2-R version 4.4.2 (2024-10-31)-1-0d5c47c21bb57e6fff68ef0cf1e77263b7fd8364f3dfe827cf5745d7ffb8c3a4"
    },
    {
        "id": 1155,
        "key": "macOS Sonoma 14.7.2-R version 4.4.0 (2024-04-24)-1-a46748d991d0f385a4b67206662ead3630e350fec232e19861a724661f8d32ea"
    },
    {
        "id": 1147,
        "key": "Ubuntu 22.04.5 LTS-R version 4.4.2 (2024-10-31)-1-3942e21f75d57e70130b5aba1366b5c20bad24e308e50eb7528285bb009c7895"
    },
    {
        "id": 1149,
        "key": "Ubuntu 22.04.5 LTS-R version 4.4.0 (2024-04-24)-1-75a8387d30a9e32c76d0e5883aedd4e6c6742016baa21d0c84e59568d8b2b50b"
    },
    {
        "id": 1143,
        "key": "Linux-poetry"
    },
    {
        "id": 1183,
        "key": "docker-dependencies-Linux-02b5e2d6aef8dd3a4ea8f51b074d3269bab5dfef8507c178d50fc75350e7aff1-dhm-run-pyrenew-h"
    }
]

The key docker-dependencies-Linux-a512b6c4b4ba68e0babeea1ad38120673c2112ebb935fdb37cda846422f06b9d-dhm-bump-default-pyrenew (which is the one generated by your job) is not present. Nonetheless, I noticed that you ran out of space for cache! For some reason, this is the message I see here (attn @damonbayer):

Approaching total cache storage limit (10.46 GB of 10 GB Used)
Least recently used caches will be automatically evicted to limit the total cache storage to 10 GB. Learn more about cache usage.

This may be a bug the happens when you reach the cache limit? I see that cfa-epinow2-pipeline cleared the cache (@amondal2?).

Start by reproducing the lookup-only cache checks from the two linked workflow runs and compare their reported keys with gh cache ls --json id,key. Investigate whether cache eviction or the 10 GB storage limit explains the false hits; done means identifying the cause and documenting or correcting the behavior with a regression test if the repository provides one.