AlmaLinux / AlmaLinux/build-system

Add the ability in bootstrap_repositories.py to resume the process from where it stopped

オープン
#265 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る
主要言語
言語のデータがありません
スター
32
フォーク
11
PR マージ指標
30日以内にマージされた PR はありません

説明

Tried to import the repos into pulp using the bootstrap_repositories.py scrip and at some point, it failed with the following error

```
INFO:repo-bootstrapper:Syncing Repository: almalinux-8-nfv i686 http://pulp/pulp/content/prod/almalinux-8-nfv-i686/ from RepositoryRemote: almalinux-8-nfv-i686 i686 http://rsync.repo.almalinux.org/almalinux/8/NFV/i686/os/...
Traceback (most recent call last):
File "/code/scripts/bootstrap_repositories.py", line 273, in
main()
File "/code/scripts/bootstrap_repositories.py", line 262, in main
sync(pulp_client.sync_rpm_repo_from_remote(
File "/usr/lib64/python3.9/functools.py", line 888, in wrapper
return dispatch(args[0].__class__)(*args, **kw)
File "/code/env/lib/python3.9/site-packages/syncer.py", line 33, in sync_co
return asyncio.get_event_loop().run_until_complete(co)
File "/usr/lib64/python3.9/asyncio/base_events.py", line 647, in run_until_complete
return future.result()
File "/code/alws/utils/pulp_client.py", line 500, in sync_rpm_repo_from_remote
result = await self.wait_for_task(task['task'])
File "/code/alws/utils/pulp_client.py", line 615, in wait_for_task
raise Exception(f'Task {str(task)} has failed')
Exception: Task {'pulp_href': '/pulp/api/v3/tasks/dcc80ace-f3ea-419e-9152-6aa7de768fe1/', 'pulp_created': '2022-07-15T09:17:29.108353Z', 'state': 'failed', 'name': 'pulp_rpm.app.tasks.synchronizing.synchronize', 'logging_cid': '97c71e1392b54b808348f7979d9efd93', 'started_at': '2022-07-15T09:17:29.224114Z', 'finished_at': '2022-07-15T09:17:29.819785Z', 'error': {'traceback': ' File "/usr/local/lib/python3.8/site-packages/pulpcore/tasking/pulpcore_worker.py", line 410, in _perform_task\n result = func(*args, **kwargs)\n File "/usr/local/lib/python3.8/site-packages/pulp_rpm/app/tasks/synchronizing.py", line 464, in synchronize\n remote_url = fetch_remote_url(remote, url)\n File "/usr/local/lib/python3.8/site-packages/pulp_rpm/app/tasks/synchronizing.py", line 298, in fetch_remote_url\n remote_url = fetch_mirror(remote)\n File "/usr/local/lib/python3.8/site-packages/pulp_rpm/app/tasks/synchronizing.py", line 250, in fetch_mirror\n result = downloader.fetch()\n File "/usr/local/lib/python3.8/site-packages/pulpcore/download/base.py", line 187, in fetch\n return done.pop().result()\n File "/usr/local/lib/python3.8/site-packages/pulpcore/download/http.py", line 271, in run\n return await download_wrapper()\n File "/usr/local/lib/python3.8/site-packages/backoff/_async.py", line 133, in retry\n ret = await target(*args, **kwargs)\n File "/usr/local/lib/python3.8/site-packages/pulpcore/download/http.py", line 256, in download_wrapper\n return await self._run(extra_data=extra_data)\n File "/usr/local/lib/python3.8/site-packages/pulp_rpm/app/downloaders.py", line 117, in _run\n self.raise_for_status(response)\n File "/usr/local/lib/python3.8/site-packages/pulp_rpm/app/downloaders.py", line 102, in raise_for_status\n response.raise_for_status()\n File "/usr/local/lib64/python3.8/site-packages/aiohttp/client_reqrep.py", line 1004, in raise_for_status\n raise ClientResponseError(\n', 'description': "404, message='Not Found', url=URL('http://rsync.repo.almalinux.org/almalinux/8/NFV/i686/os')"}, 'worker': '/pulp/api/v3/workers/a629915a-b048-4b35-8198-11baa7659196/', 'parent_task': None, 'child_tasks': [], 'task_group': None, 'progress_reports': [], 'created_resources': [], 'reserved_resources_record': ['/pulp/api/v3/repositories/rpm/rpm/01649371-8733-4f17-b2e2-7b9c2f40f17d/', 'shared:/pulp/api/v3/remotes/rpm/rpm/f2eb53aa-6842-4b9f-90a2-62b770779fee/']} has failed
```

The problem was already fixed upstream. Had to run the script again, which takes some time because it starts all over again. It would be great if we can bring to this script the ability to resume from where it failed.

コントリビューションガイド

コントリビューションガイドを開く

評価

この issue はまだ評価されていません。

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。