mozilla-releng / mozilla-releng/scriptworker-scripts

beetmover doesn't skip GCS uploads when it claims to

Aperta
#676 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

beetmoverscript
Lingua principale
Python
Stelle
16
Fork
38
Merge medio
1g 7h
PR unite (30g)
14

Descrizione

set_gcs_client has a number of error conditions that result in an early return, and context.gcs_client being unset. The messages given claim that GCS uploads will be skipped, but instead it continues on and ends up failing. For example:

2023-02-23 01:22:56,468 - beetmoverscript.gcloud - WARNING - GCS bucket <Bucket: mozilla-releng-dep-partner> doesn't exist. Skipping GCS uploads.
2023-02-23 01:22:56,470 - beetmoverscript.task - DEBUG - Loading release_props from task's payload: {'appName': 'Firefox', 'appVersion': '112.0', 'branch': 'try', 'buildid': '20230222192533', 'hashType': 'sha512', 'platform': 'linux'}
2023-02-23 01:22:56,470 - beetmoverscript.task - INFO - Action types: ['push-to-partner']
<snip>
2023-02-23 01:22:56,716 - beetmoverscript.task - DEBUG - Loading release_props from task's payload: {'appName': 'Firefox', 'appVersion': '112.0', 'branch': 'try', 'buildid': '20230222192533', 'hashType': 'sha512', 'platform': 'linux'}
2023-02-23 01:22:59,227 - beetmoverscript.script - INFO - put /app/workdir/cot/XMhpfS5_T4aYb2qrypIxyQ/releng/partner/test/test-002/win32/he/target.installer.exe: 200
2023-02-23 01:22:59,228 - charset_normalizer - DEBUG - Encoding detection on empty bytes, assuming utf_8 intention.
2023-02-23 01:22:59,413 - beetmoverscript.script - INFO - put /app/workdir/cot/XMhpfS5_T4aYb2qrypIxyQ/releng/partner/test/test-002/win64/en-CA/target.installer.exe: 200
2023-02-23 01:22:59,413 - charset_normalizer - DEBUG - Encoding detection on empty bytes, assuming utf_8 intention.
2023-02-23 01:22:59,432 - beetmoverscript.script - INFO - put /app/workdir/cot/XMhpfS5_T4aYb2qrypIxyQ/releng/partner/test/test-002/win64/he/target.installer.exe: 200
2023-02-23 01:22:59,432 - charset_normalizer - DEBUG - Encoding detection on empty bytes, assuming utf_8 intention.
2023-02-23 01:22:59,466 - beetmoverscript.script - INFO - put /app/workdir/cot/XMhpfS5_T4aYb2qrypIxyQ/releng/partner/test/test-002/win32/en-CA/target.installer.exe: 200
2023-02-23 01:22:59,466 - charset_normalizer - DEBUG - Encoding detection on empty bytes, assuming utf_8 intention.
Traceback (most recent call last):
  File "/app/bin/beetmoverscript", line 8, in <module>
    sys.exit(main())
  File "/app/lib/python3.9/site-packages/beetmoverscript/script.py", line 683, in main
    client.sync_main(async_main, config_path=config_path, default_config=default_config, should_validate_task=False)
  File "/app/lib/python3.9/site-packages/scriptworker/client.py", line 166, in sync_main
    loop.run_until_complete(_handle_asyncio_loop(async_main, context))
  File "/usr/local/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
    return future.result()
  File "/app/lib/python3.9/site-packages/scriptworker/client.py", line 205, in _handle_asyncio_loop
    await async_main(context)
  File "/app/lib/python3.9/site-packages/beetmoverscript/script.py", line 335, in async_main
    await action_map[context.action](context)
  File "/app/lib/python3.9/site-packages/beetmoverscript/script.py", line 168, in push_to_partner
    await move_partner_beets(context, mapping_manifest)
  File "/app/lib/python3.9/site-packages/beetmoverscript/script.py", line 485, in move_partner_beets
    await await_and_raise_uploads(cloud_uploads, context.config["clouds"], context.bucket)
  File "/app/lib/python3.9/site-packages/beetmoverscript/utils.py", line 333, in await_and_raise_uploads
    await raise_future_exceptions(cloud_uploads[cloud])
  File "/app/lib/python3.9/site-packages/scriptworker/utils.py", line 409, in raise_future_exceptions
    succeeded_results, _ = await _process_future_exceptions(tasks, raise_at_first_error=True)
  File "/app/lib/python3.9/site-packages/scriptworker/utils.py", line 443, in _process_future_exceptions
    raise exc
  File "/app/lib/python3.9/site-packages/beetmoverscript/gcloud.py", line 92, in upload_to_gcs
    bucket = Bucket(context.gcs_client, name=bucket_name)
AttributeError: 'Context' object has no attribute 'gcs_client'

Guida per i contributori

Nessuna guida per i contributori indicizzata per questo repository

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Direzione di ricerca

Inizia in beetmoverscript/gcloud.py, in set_gcs_client e upload_to_gcs, poi segui i relativi chiamanti in beetmoverscript/script.py e beetmoverscript/utils.py. Riproduci o traccia il percorso del client mancante e verifica che ogni condizione di ritorno anticipato salti gli upload su GCS senza generare un AttributeError.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
google-cloud, python
Ambito
cloud
Tipo di issue
Bug
Difficoltà
2/5
Tempo stimato
1-3 ore
Stato di attività
Ferma
Chiarezza
Specificata chiaramente
Idoneità per principianti
45/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.