Logs failing to flush in `Logdna::Client` due to concurrency issue in rails
Nessuno ha ancora preso questa issue.
- Lingua principale
- Ruby
- Stelle
- 19
- Fork
- 18
- Metriche di merge delle PR
- Nessuna PR unita negli ultimi 30g
Descrizione
Found that any uses of Logdna::Client within the boot sequence of rails results in subsequent logs never flushing.
For background, using logdna 1.5.0 w/ rails 6.1.7.5 on ruby 3.2.2. Configuring Logdna::Ruby following the instructions in TAMING RUBY ON RAILS LOGGING WITH LOGRAGE AND MEZMO in config/application.rb. As long as Rails.logger isn't used within the subsequent application setup steps, then logging works as expected. But if any other subsequent step of the application setup process attempts to log, then all subsequent logs fail to be emitted.
Tracked this down to Logdna::Client#schedule_flush. By adding logging, I found that the associated Concurrent::ScheduledTask in @scheduled_flush remains in the pending state indefinitely. Further logging shows that the ScheduledTask is associated with a Concurrent::CachedThreadPool that contains a single thread and that thread has is marked as dead.
Guessing that as part of the rails boot up process, the default executor is terminated and therefore the @scheduled_flush never completes. Not that using the @work_thread_pool thread pool doesn't fix the issue; the same behavior is observed. By not logging anything during rails configurations/boot, a ScheduledTask isn't created while the application is still booting, and therefore this issue doesn't occur.
I think this issue could be addressed by tracking the time at which @scheduled_flush is created and then in subsequent invocations of schedule_flush checking if it has been pending far longer than expected. If it is sufficiently stale, then the outstanding ScheduledTask would be cancelled and a new one created. There may be more robust/elegant ways to address this because I'm not particularly familiar with ruby/rails thread management.
Guida per i contributori
Apri la guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Direzione di ricerca
Inizia da Logdna::Client#schedule_flush e analizza come @scheduled_flush e @work_thread_pool utilizzano le attività pianificate di Concurrent durante l'avvio di Rails. Riproduci il problema con la configurazione descritta di Rails 6.1.7.5 e Ruby 3.2.2, includendo una chiamata di log da config/application.rb; il lavoro è completato quando i log continuano a essere scaricati dopo l'avvio, anche quando vengono prodotti log di configurazione.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Valutazione
- Stack tecnologico
- rails, ruby
- Ambito
- backend, observability-sre
- Tipo di issue
- Bug
- Difficoltà
- 4/5
- Tempo stimato
- 3-5 giorni
- Stato di attività
- Ferma
- Chiarezza
- Abbastanza chiara
- Idoneità per principianti
- 35/100