anchore / anchore/syft

File digests not generated for directory scans despite correct configuration

Open
#4,564 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
9.6k
Forks
954
Avg merge
23h 27m
Merged PRs (30d)
48

Description

**What happened**:

File digests are not being generated when scanning directories, even with proper configuration. The digest cataloger processes 0 files for directory scans, while the same configuration works correctly for container image scans.

Debug output shows:
```
[0000] DEBUG file metadata cataloger processed 606 files
[0000] DEBUG file digests cataloger processed 0 files
```

The configuration is being loaded correctly (verified in `descriptor.configuration.files`), but no digests are computed.

**What you expected to happen**:

File digests should be generated for directory scans when `file.metadata.selection` is set to `all` and `file.metadata.digests` is configured, just as they are for container image scans.

**Steps to reproduce the issue**:

1. Create a test directory with some files:
```bash
cd /tmp && mkdir -p syft-dir-test/subdir
cd /tmp/syft-dir-test
echo "This is test file 1" > file1.txt
echo "This is test file 2" > subdir/file2.txt
```

2. Create a configuration file `config-all.yaml`:
```yaml
file:
metadata:
selection: all
digests:
- sha256
- sha512
```

3. Run Syft with debug logging:
```bash
syft scan dir:. -c config-all.yaml -o syft-json -vv > output-all.json 2>&1
```

4. Check the debug output - you'll see:
```
[0000] DEBUG file metadata cataloger processed 606 files
[0000] DEBUG file digests cataloger processed 0 files
```

5. Verify no digests in output:
```bash
jq '[.files[] | select(.digests != null)] | length' output-all.json
# Returns: 0
```

**Contrast with working container image scan:**

The same configuration works correctly for container images:
```bash
syft scan alpine:latest -c config-all.yaml -o syft-json > alpine-output.json
jq '[.files[] | select(.digests != null)] | length' alpine-output.json
# Returns: 83 (files with digests)
```

**Anything else we need to know?**:

**Root Cause Analysis:**

Based on code investigation, the issue appears to be in `syft/file/cataloger/internal/all_regular_files.go`. The `AllRegularFiles()` function is returning an empty list for directory scans, even though:

1. The file metadata cataloger successfully processes 606 files
2. The configuration is loaded correctly (verified in SBOM output)
3. The digest cataloger is invoked but receives 0 files to process

The `AllRegularFiles()` function:
- Calls `resolver.AllLocations(ctx)` to get all file locations
- For each location, calls `resolver.FilesByPath(location.RealPath)` to resolve it
- For each resolved location, calls `resolver.FileMetadataByLocation(resolvedLocation)` to get metadata
- Filters for regular files only

Since the metadata cataloger works but the digest cataloger doesn't, there's likely a difference in how they access files or a bug in `AllRegularFiles()` when used with directory resolvers.

**Code References:**
- `internal/task/file_tasks.go` - `newFileDigestCatalogerTask()` creates the digest task
- `syft/file/cataloger/filedigest/cataloger.go` - Line 41 calls `AllRegularFiles()` when coordinates is empty
- `syft/file/cataloger/internal/all_regular_files.go` - The function that returns 0 files for directory scans

**Environment**:
- Output of `syft version`:
```
Application: syft
Version: 1.40.1
BuildDate: 2025-01-15T19:23:14Z
GitCommit: 386ef842d99a72027fb5fd1085fde87883640eaf
GitDescription: v1.40.1
Platform: darwin/arm64
GoVersion: go1.24.3
Compiler: gc
SchemaVersion: 16.1.2
```
- OS: macOS 15.2.1 (also reproduced on Linux)

Contributor guide

Open the contributing guide

Research direction

Start with syft/file/cataloger/internal/all_regular_files.go and compare its directory-scan behavior with the metadata cataloger. Trace the digest task from internal/task/file_tasks.go through syft/file/cataloger/filedigest/cataloger.go, then reproduce with the provided config and directory scan command. Done means directory scans emit configured SHA-256 and SHA-512 digests in the SBOM output.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
cli
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.