'NoneType' object has no attribute 'artifact_set_sha256' when syncing a recently updated deb remote
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 71
- Forks
- 91
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 22
Description
Version
Installed and setup via ansible (pulp_installer)
{
"component": "core",
"version": "3.21.3",
"package": "pulpcore"
},
{
"component": "deb",
"version": "2.20.0",
"package": "pulp_deb"
}
Describe the bug
It looks like there is an issue comparing the indices for a remote when syncing a repository after the remote has updated. When running a sync job, it returns with this error:
.Error: Task /pulp/api/v3/tasks/28fcc35c-15cf-4e00-be84-98041c6dd7f2/ failed: ''NoneType' object has no attribute 'artifact_set_sha256''
The stack trace for the error is:
File "/pulp/pulpenv/lib64/python3.9/site-packages/pulpcore/tasking/pulpcore_worker.py", line 452, in _perform_task
result = func(*args, **kwargs)
File "/pulp/pulpenv/lib64/python3.9/site-packages/pulp_deb/app/tasks/synchronizing.py", line 170, in synchronize
DebDeclarativeVersion(first_stage, repository, mirror=mirror).create()
File "/pulp/pulpenv/lib64/python3.9/site-packages/pulpcore/plugin/stages/declarative_version.py", line 161, in create
loop.run_until_complete(pipeline)
File "/usr/lib64/python3.9/asyncio/base_events.py", line 647, in run_until_complete
return future.result()
File "/pulp/pulpenv/lib64/python3.9/site-packages/pulpcore/plugin/stages/api.py", line 225, in create_pipeline
await asyncio.gather(*futures)
File "/pulp/pulpenv/lib64/python3.9/site-packages/pulpcore/plugin/stages/api.py", line 43, in __call__
await self.run()
File "/pulp/pulpenv/lib64/python3.9/site-packages/pulp_deb/app/tasks/synchronizing.py", line 544, in run
await asyncio.gather(
File "/pulp/pulpenv/lib64/python3.9/site-packages/pulp_deb/app/tasks/synchronizing.py", line 679, in _handle_distribution
await asyncio.gather(*sub_tasks)
File "/pulp/pulpenv/lib64/python3.9/site-packages/pulp_deb/app/tasks/synchronizing.py", line 763, in _handle_component
await asyncio.gather(*pending_tasks)
File "/pulp/pulpenv/lib64/python3.9/site-packages/pulp_deb/app/tasks/synchronizing.py", line 861, in _handle_package_index
if previous_package_index.artifact_set_sha256 == package_index.artifact_set_sha256:
To Reproduce
It may be difficult to reproduce. This is only happening on the microsoft repositories. We are syncing arch amd64 for jammy, focal, and bionic. In order to reproduce it you need to have a previous version of the repository/release. If you setup a brand new remote and repository, it doesn't encounter this issue when syncing, since it has nothing to compare from.
These are the basic settings
pulp deb remote create --name "microsoft_deb_focal" --architecture amd64 --distribution focal --url "https://packages.microsoft.com/ubuntu/20.04/prod"
pulp deb repository create --description "Microsoft focal repo" --remote microsoft_deb_focal --name repo_microsoft_deb_focal
pulp deb repository sync --name repo_microsoft_deb_focal
Expected behavior
The repository should sync and pull in the new updates without any issues.
Additional context
The remote microsoft repos we're mirror'ing were updated several days ago. The day they were updated is when our sync's started to fail. It's only for those repos. I am uncertain if they configured something incorrectly in their release, or it's simply a bug with the plugin. The timing is odd nonetheless. These are the URLs:
- https://packages.microsoft.com/ubuntu/18.04/prod
- https://packages.microsoft.com/ubuntu/20.04/prod
- https://packages.microsoft.com/ubuntu/22.04/prod
if it's of any relevance, here is the remote config for one of the above. They're identical aside from name and URL
{
"pulp_href": "/pulp/api/v3/remotes/deb/apt/819cf3bf-4559-4027-8ed6-fa2c27eb462b/",
"pulp_created": "2022-12-09T22:11:46.085397Z",
"name": "deb_remote_microsoft_focal",
"url": "https://packages.microsoft.com/ubuntu/20.04/prod",
"ca_cert": null,
"client_cert": null,
"tls_validation": true,
"proxy_url": null,
"pulp_labels": {},
"pulp_last_updated": "2023-02-06T14:53:20.757223Z",
"download_concurrency": 2,
"max_retries": 10,
"policy": "on_demand",
"total_timeout": 1200.0,
"connect_timeout": null,
"sock_connect_timeout": null,
"sock_read_timeout": 60.0,
"headers": null,
"rate_limit": null,
"distributions": "focal",
"components": null,
"architectures": "amd64",
"sync_sources": false,
"sync_udebs": false,
"sync_installer": false,
"gpgkey": null,
"ignore_missing_package_indices": false
}
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in pulp_deb/app/tasks/synchronizing.py at _handle_package_index, around line 861, and trace how the previous package index is selected during synchronization. Reproduce with an existing Microsoft Ubuntu repository and an updated remote, then verify that synchronization completes and pulls the new updates instead of failing on the missing previous index.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100