File digests not generated for directory scans despite correct configuration
- Dominant language
- Go
- Stars
- 9.6k
- Forks
- 954
- Avg merge
- 23h 27m
- Merged PRs (30d)
- 48
Description
**What happened**:
File digests are not being generated when scanning directories, even with proper configuration. The digest cataloger processes 0 files for directory scans, while the same configuration works correctly for container image scans.
Debug output shows:
```
[0000] DEBUG file metadata cataloger processed 606 files
[0000] DEBUG file digests cataloger processed 0 files
```
The configuration is being loaded correctly (verified in `descriptor.configuration.files`), but no digests are computed.
**What you expected to happen**:
File digests should be generated for directory scans when `file.metadata.selection` is set to `all` and `file.metadata.digests` is configured, just as they are for container image scans.
**Steps to reproduce the issue**:
1. Create a test directory with some files:
```bash
cd /tmp && mkdir -p syft-dir-test/subdir
cd /tmp/syft-dir-test
echo "This is test file 1" > file1.txt
echo "This is test file 2" > subdir/file2.txt
```
2. Create a configuration file `config-all.yaml`:
```yaml
file:
metadata:
selection: all
digests:
- sha256
- sha512
```
3. Run Syft with debug logging:
```bash
syft scan dir:. -c config-all.yaml -o syft-json -vv > output-all.json 2>&1
```
4. Check the debug output - you'll see:
```
[0000] DEBUG file metadata cataloger processed 606 files
[0000] DEBUG file digests cataloger processed 0 files
```
5. Verify no digests in output:
```bash
jq '[.files[] | select(.digests != null)] | length' output-all.json
# Returns: 0
```
**Contrast with working container image scan:**
The same configuration works correctly for container images:
```bash
syft scan alpine:latest -c config-all.yaml -o syft-json > alpine-output.json
jq '[.files[] | select(.digests != null)] | length' alpine-output.json
# Returns: 83 (files with digests)
```
**Anything else we need to know?**:
**Root Cause Analysis:**
Based on code investigation, the issue appears to be in `syft/file/cataloger/internal/all_regular_files.go`. The `AllRegularFiles()` function is returning an empty list for directory scans, even though:
1. The file metadata cataloger successfully processes 606 files
2. The configuration is loaded correctly (verified in SBOM output)
3. The digest cataloger is invoked but receives 0 files to process
The `AllRegularFiles()` function:
- Calls `resolver.AllLocations(ctx)` to get all file locations
- For each location, calls `resolver.FilesByPath(location.RealPath)` to resolve it
- For each resolved location, calls `resolver.FileMetadataByLocation(resolvedLocation)` to get metadata
- Filters for regular files only
Since the metadata cataloger works but the digest cataloger doesn't, there's likely a difference in how they access files or a bug in `AllRegularFiles()` when used with directory resolvers.
**Code References:**
- `internal/task/file_tasks.go` - `newFileDigestCatalogerTask()` creates the digest task
- `syft/file/cataloger/filedigest/cataloger.go` - Line 41 calls `AllRegularFiles()` when coordinates is empty
- `syft/file/cataloger/internal/all_regular_files.go` - The function that returns 0 files for directory scans
**Environment**:
- Output of `syft version`:
```
Application: syft
Version: 1.40.1
BuildDate: 2025-01-15T19:23:14Z
GitCommit: 386ef842d99a72027fb5fd1085fde87883640eaf
GitDescription: v1.40.1
Platform: darwin/arm64
GoVersion: go1.24.3
Compiler: gc
SchemaVersion: 16.1.2
```
- OS: macOS 15.2.1 (also reproduced on Linux)
Contributor guide
Research direction
Start with syft/file/cataloger/internal/all_regular_files.go and compare its directory-scan behavior with the metadata cataloger. Trace the digest task from internal/task/file_tasks.go through syft/file/cataloger/filedigest/cataloger.go, then reproduce with the provided config and directory scan command. Done means directory scans emit configured SHA-256 and SHA-512 digests in the SBOM output.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- cli
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100