python-jsonschema / python-jsonschema/check-jsonschema

Support ETag for cache validation when Last-Modified is unavailable

Open
#668 9 comments 0 reactions 1 assignee View on GitHub

@sirosen is already working on this.

Since Mar 26, 2026.

enhancement
Dominant language
Python
Stars
341
Forks
71
Avg merge
15d 20h
Merged PRs (30d)
4

Description

Summary

When a remote schema endpoint provides an ETag header but no Last-Modified header, check-jsonschema currently treats the cache as always-fresh, leading to stale schemas being served indefinitely.

Current Behavior

The cache hit logic in cachedownloader.py relies solely on Last-Modified:

def _cache_hit(cachefile: str, response: requests.Response) -> bool:
    if not os.path.exists(cachefile):
        return False
    local_mtime = os.path.getmtime(cachefile)
    remote_mtime = _lastmod_from_response(response)
    return local_mtime >= remote_mtime

When Last-Modified is missing, _lastmod_from_response() returns 0.0:

def _lastmod_from_response(response: requests.Response) -> float:
    try:
        return calendar.timegm(
            time.strptime(response.headers["last-modified"], _LASTMOD_FMT)
        )
    except (OverflowError, ValueError, LookupError):
        return 0.0

Since any cached file's mtime is >= 0.0, the cache never invalidates.

Real-World Impact

Mergify's schema endpoint (https://docs.mergify.com/mergify-configuration-schema.json) provides ETag but not Last-Modified:

$ curl -sI https://docs.mergify.com/mergify-configuration-schema.json | grep -E '^(cache-control|etag|last-modified):'
cache-control: public, max-age=600, no-transform
etag: "afd19c79c195c2e76f1d37bd12421d88"

When Mergify adds new config options, users get false validation failures until they manually clear ~/.cache/check_jsonschema/.

I've filed Mergifyio/mergify#5161 requesting they add Last-Modified, but this is likely a common pattern for CDN-served content (Cloudflare in their case).

Suggested Enhancement

Support ETag as a fallback when Last-Modified is unavailable:

  1. Store the ETag value alongside cached files (e.g., in a .etag sidecar file or a metadata store)
  2. On subsequent requests, send If-None-Match: <stored-etag> header
  3. If server returns 304 Not Modified, treat as cache hit
  4. If server returns 200 with new ETag, update cache and stored ETag

This follows standard HTTP caching semantics where clients should support both Last-Modified/If-Modified-Since and ETag/If-None-Match.

Alternatives Considered

  • Require servers to set Last-Modified: Not always under user control, especially for third-party schemas
  • Use Cache-Control: max-age: Already present in some responses (e.g., max-age=600), could be used as a TTL, though this would mean re-downloading more frequently than necessary when content hasn't changed

References

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.