huggingface / huggingface/swift-transformers
Offline-mode metadata lookup fails when downloadBase contains '..' path components
- Dominant language
- Swift
- Stars
- 1.4k
- Forks
- 209
- Avg merge
- 2d 7h
- Merged PRs (30d)
- 4
Description
**swift-transformers version**: 0.1.24 (revision `f000aa7aec0e78acd0211685e4094e1fca84cd8b`)
**Platform**: macOS 26 (Apple Silicon)
## Summary
When `HubApi(downloadBase: someURL, useOfflineMode: true)` is constructed with a `downloadBase` whose path contains `..` components (e.g. `/Users/me/project/foo/../../resources/tokenizer`), the offline-mode snapshot loader fails with `EnvironmentError.offlineModeError("Metadata not available for ")` even when the cache layout is complete and the metadata markers exist on disk at the locations the loader would compute *after* canonicalizing.
Passing the same directory with a canonicalized absolute path (no `..`) succeeds.
## Root cause
In `Sources/Hub/HubApi.swift`, the `snapshot(...)` function (around line 538–565) walks files via `FileManager.enumerator(at: repoDestination, ...)`. The enumerator returns `URL`s whose `.path` is already canonicalized — `..` components are resolved away by the OS.
For each enumerated file, the metadata path is built via:
```swift
let metadataPath = URL(
fileURLWithPath: fileUrl.path.replacingOccurrences(
of: repoDestination.path,
with: repoMetadataDestination.path
) + ".metadata"
)
```
`fileUrl.path` is canonical (no `..`). `repoDestination.path` is whatever was passed in by the caller via `downloadBase`, which preserves `..` if present. The substring replacement therefore fails to match, the resulting `metadataPath` is wrong, `readDownloadMetadata` returns nil because the file doesn't exist at the wrong path, and the loader throws `Metadata not available`.
## Repro
Minimal Swift package using swift-transformers 0.1.24:
```swift
import Foundation
import Hub
import Tokenizers
// A well-formed offline cache at /tmp/cache-root/models/BAAI/bge-base-en-v1.5/
// produced by huggingface_hub.snapshot_download + writeDownloadMetadata equivalents.
// Path WITH '..' components — fails
let badBase = URL(fileURLWithPath: "/tmp/cache-root/sub/../")
let hub1 = HubApi(downloadBase: badBase, useOfflineMode: true)
_ = try await AutoTokenizer.from(pretrained: "BAAI/bge-base-en-v1.5", hubApi: hub1)
// → throws offlineModeError("Metadata not available for tokenizer_config.json")
// Same cache directory, canonical path — succeeds
let goodBase = URL(fileURLWithPath: "/tmp/cache-root")
let hub2 = HubApi(downloadBase: goodBase, useOfflineMode: true)
_ = try await AutoTokenizer.from(pretrained: "BAAI/bge-base-en-v1.5", hubApi: hub2)
// → loads fine
```
## Suggested fix
Canonicalize `repoDestination.path` (and/or `repoMetadataDestination.path`) before the substring replacement, or build the metadata path by URL appending rather than string substitution. Something like:
```swift
let repoDestinationCanonical = repoDestination.standardizedFileURL
let repoMetadataDestinationCanonical = repoMetadataDestination.standardizedFileURL
// ... use canonical paths in the substitution
```
Or, more robustly, compute the metadata path via URL operations rather than path-string surgery:
```swift
let relPath = fileUrl.path.replacingOccurrences(of: repoDestination.standardizedFileURL.path + "/", with: "")
let metadataPath = repoMetadataDestination.appendingPathComponent(relPath + ".metadata")
```
## Workaround
Callers can canonicalize the `downloadBase` URL before passing it to `HubApi`:
```swift
let hub = HubApi(downloadBase: userSuppliedURL.standardizedFileURL, useOfflineMode: true)
```
We've adopted this defensive workaround in our downstream project (verveguy/liminis#815) but the underlying behavior is surprising — silent metadata-lookup failure on a path with `..` is a meaningful footgun for callers building integrations.
## Related
Discovered during end-to-end validation of a Liminis embedding sidecar (BGE-base-en-v1.5 via CoreML). The cache layout was generated correctly by a `huggingface_hub.snapshot_download`-based script plus per-file `.metadata` markers written via the documented 3-line format. Standalone repro with the canonical path succeeds; sidecar (which passed a `\$PWD/../..` constructed env-var path) fails.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start in Sources/Hub/HubApi.swift, around snapshot(...) lines 538–565, and reproduce the issue with the minimal Swift package example using a downloadBase containing '..'. The work is done when offline-mode metadata lookup succeeds for both the non-canonical and canonical cache paths, without the Metadata not available error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- swift
- Domain
- api
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100