Syft's dependency resolution for Python packages differs from pipdeptree's output
- Dominant language
- Go
- Stars
- 9.6k
- Forks
- 962
- Avg merge
- 23h 27m
- Merged PRs (30d)
- 48
Description
[python_jupyterhub_rvm_report.json](https://github.com/user-attachments/files/19228672/python_jupyterhub_rvm_report.json)
[pipdeptree_report.json](https://github.com/user-attachments/files/19228673/pipdeptree_report.json)
**What happened**:
Syft's dependency resolution for Python packages differs from pipdeptree's output, particularly in how it handles package names and version constraints. For example, with Alembic 1.13.3:
Pipdeptree shows:
- Mako (1.3.5)
- SQLAlchemy (>=1.3.0)
- typing_extensions (>=4)
While Syft shows different dependencies:
- importlib-metadata (8.0.0)
- importlib-resources (6.4.0)
The discrepancy appears to be due to two issues:
1. Package name normalization is not being performed in the dependency resolver
2. Version constraint parsing is incomplete in the Python dependency cataloger
**What you expected to happen**:
Syft should report the same dependencies as pipdeptree, correctly handling:
- Case-insensitive package name matching
- All version constraint formats (including spaces before version numbers)
- Proper extraction of package names from requirement strings
**Steps to reproduce the issue**:
1. Launch jupyterhub/jupyterhub container:
```bash
kubectl exec -it -n e2e-rvm-test e2e-rvm-jupyterhub-container -- bash -c "pip install pipdeptree"
kubectl exec -it -n e2e-rvm-test e2e-rvm-jupyterhub-container -- bash -c "pipdeptree --json" > pipdeptree_report.json
```
2 .Run Syft scan on the same container
3. Compare dependency outputs for Alembic package
**Code Changes Needed**:
1. In resolver.go - Normalize resource names:
```
// resource needs to be normalized here (to lowercase)
for providingPkgID := range pkgsProvidingResource[strings.ToLower(resource)] {
```
2. In dependency.go - Fix version constraint parsing:
```
// the split needs to include ><=
return strings.TrimSpace(internal.SplitAny(s, "(;>=<")[0])
```
**Test case demonstrating the issue**:
```
func TestExtractPackageName(t *testing.T) {
tests := []struct {
name string
packageName string
expected string
}{
{
name: "Package with parentheses version",
packageName: "soupsieve (>1.2)",
expected: "soupsieve",
},
{
name: "Package with space version",
packageName: "SQLAlchemy >=1.3.0",
expected: "SQLAlchemy",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
assert.Equal(t, tt.expected, extractPackageName(tt.packageName))
})
}
}
```
**Environment**:
Output of syft version: v1.19.0
OS: Linux (tested on jupyterhub/jupyterhub container)
Python package: Alembic 1.13.3
Contributor guide
Research direction
Start in resolver.go and dependency.go, focusing on resource-name normalization and extractPackageName's handling of version constraints. Run the provided TestExtractPackageName cases and compare the Alembic dependency output with pipdeptree; done means normalized package names and the listed constraint formats produce matching dependencies.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, python
- Domain
- cli, tooling
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 35/100