apache / apache/airflow

logging issue when running celery worker in debug mode via breeze

Open
#56,865 3 comments 0 reactions 1 assignee Claimed by @rawwar View on GitHub
area:logging kind:bug provider:celery
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 10h
Merged PRs (30d)
483

Description

### Body

While i was debugging through the celery worker code, i noticed, there's a logging issue. Every log message seems to be have below traceback

```
Arguments: ('http://[::]:8793',)
WARNING: ASGI app factory detected. Using it, but please consider setting the --factory flag explicitly.
INFO: Started server process [371]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on http://:8793 (Press CTRL+C to quit)
--- Logging error ---
Traceback (most recent call last):
File "/usr/python/lib/python3.10/logging/__init__.py", line 1103, in emit
stream.write(msg + self.terminator)
File "/usr/python/lib/python3.10/site-packages/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_io.py", line 40, in write
r.write(s)
TypeError: a bytes-like object is required, not 'str'
Call stack:
File "/usr/python/bin/debugpy", line 10, in
sys.exit(main())
File "/usr/python/lib/python3.10/site-packages/debugpy/server/cli.py", line 508, in main
run()
File "/usr/python/lib/python3.10/site-packages/debugpy/server/cli.py", line 391, in run_module
run_module_as_main(options.target, alter_argv=True)
File "/usr/python/lib/python3.10/site-packages/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 228, in _run_module_as_main
return _run_code(code, main_globals, None, "__main__", mod_spec)
File "/usr/python/lib/python3.10/site-packages/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 118, in _run_code
exec(code, run_globals)
File "/opt/airflow/airflow-core/src/airflow/__main__.py", line 59, in
main()
File "/opt/airflow/airflow-core/src/airflow/__main__.py", line 55, in main
args.func(args)
File "/opt/airflow/airflow-core/src/airflow/cli/cli_config.py", line 49, in command
return func(*args, **kwargs)
File "/opt/airflow/airflow-core/src/airflow/utils/cli.py", line 115, in wrapper
return f(*args, **kwargs)
File "/opt/airflow/providers/celery/src/airflow/providers/celery/cli/celery_command.py", line 66, in wrapper
providers_configuration_loaded(func)(*args, **kwargs)
File "/opt/airflow/airflow-core/src/airflow/utils/providers_configuration_loader.py", line 54, in wrapped_function
return func(*args, **kwargs)
File "/opt/airflow/providers/celery/src/airflow/providers/celery/cli/celery_command.py", line 281, in worker
_run_command_with_daemon_option(
File "/opt/airflow/providers/celery/src/airflow/providers/celery/cli/celery_command.py", line 52, in _run_command_with_daemon_option
run_command_with_daemon_option(*args, **kwargs)
File "/opt/airflow/airflow-core/src/airflow/cli/commands/daemon_utils.py", line 86, in run_command_with_daemon_option
callback()
File "/opt/airflow/providers/celery/src/airflow/providers/celery/cli/celery_command.py", line 273, in run_celery_worker
with _serve_logs(skip_serve_logs), _run_stale_bundle_cleanup():
File "/usr/python/lib/python3.10/contextlib.py", line 135, in __enter__
return next(self.gen)
File "/opt/airflow/providers/celery/src/airflow/providers/celery/cli/celery_command.py", line 141, in _run_stale_bundle_cleanup
log.info("starting stale bundle cleanup process")
Message: 'starting stale bundle cleanup process'
Arguments: ()
```

Breeze command:

```
breeze start-airflow --executor CeleryExecutor --backend postgres --debug celery-worker
```

this uses `debugpy` to start the worker

### Committer

- [x] I acknowledge that I am a maintainer/committer of the Apache Airflow project.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.