ruby-concurrency / ruby-concurrency/concurrent-ruby

Passenger workers hang / segfault with Ruby 2.5 / concurrent-ruby 1.1.6

Aperta
#871 3 commenti 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

bug
Lingua principale
Ruby
Stelle
5.8k
Fork
420
Merge medio
20h 45m
PR unite (30g)
4

Descrizione

Thank you for all your hard work on the latest 1.1.6 release.

I recently upgraded our app gems and started noticing the passenger workers would become unresponsive (request queue = 100) and not shutdown (normal mem/no CPU usage) after ~12-24 hours. Inspecting the error logs, I found some segfaults that appeared related to the finalization code present in 1.1.6 and a possible MRI bug. Rolling back this single gem to 1.1.5 resolved the issue.

Phusion Passenger 5.0.24
Rails 6.0.0
* Operating system:                Ubuntu 14.04.6 LTS
* Ruby implementation:             ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux]
* `concurrent-ruby` version:      1.1.6
* `concurrent-ruby-ext` installed: no
* `concurrent-ruby-edge` used:     no
Initial segfault data (let me know if the rest would be helpful)
App 26150 stderr: /opt/production_engine/shared/bundled_gems/ruby/2.5.0/gems/concurrent-ruby-1.1.6/lib/concurrent-ruby/concurrent/atomic/ruby_thread_local_var.rb:113:
App 26150 stderr: [BUG]
App 26150 stderr: Segmentation fault at 0x0000000000000050
App 26150 stderr:
App 26150 stderr: ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux]
App 26150 stderr:
App 26150 stderr: -- Control frame information -----------------------------------------------
App 26150 stderr: c:0003
App 26150 stderr: p:----
App 26150 stderr: s:0011
App 26150 stderr: e:000010
App 26150 stderr: CFUNC
App 26150 stderr:  :push
App 26150 stderr:
App 26150 stderr: c:0002
App 26150 stderr: p:0017
App 26150 stderr: s:0006
App 26150 stderr: e:000005
App 26150 stderr: BLOCK
App 26150 stderr:  /opt/production_engine/shared/bundled_gems/ruby/2.5.0/gems/concurrent-ruby-1.1.6/lib/concurrent-ruby/concurrent/atomic/ruby_thr
App 26150 stderr:  [FINISH]
App 26150 stderr:
App 26150 stderr: c:0001
App 26150 stderr: p:0000
App 26150 stderr: s:0003
App 26150 stderr: E:0022e0
App 26150 stderr: (none)
App 26150 stderr:  [FINISH]
App 26150 stderr:
App 26150 stderr:
App 26150 stderr: -- Ruby level backtrace information ----------------------------------------
App 26150 stderr: /opt/production_engine/shared/bundled_gems/ruby/2.5.0/gems/concurrent-ruby-1.1.6/lib/concurrent-ruby/concurrent/atomic/ruby_thread_local_var.rb:113:in `block in thread_local_finalizer'
App 26150 stderr: /opt/production_engine/shared/bundled_gems/ruby/2.5.0/gems/concurrent-ruby-1.1.6/lib/concurrent-ruby/concurrent/atomic/ruby_thread_local_var.rb:113:in `push'
App 26150 stderr:
App 26150 stderr: -- Machine register context ------------------------------------------------
App 26150 stderr:  RIP: 0x00007f3d1c008d96
App 26150 stderr:  RBP: 0x0000000000000000
App 26150 stderr:  RSP: 0x00007ffed1b32bb0
App 26150 stderr:
App 26150 stderr:  RAX: 0x00007f3d187d61f8
App 26150 stderr:  RBX: 0x00007f3d1c0084e4
App 26150 stderr:  RCX: 0x00007f3d1e742ba8
App 26150 stderr:
App 26150 stderr:  RDX: 0x00007f3d1c0084ec
App 26150 stderr:  RDI: 0x0000000000000000
App 26150 stderr:  RSI: 0x00007f3d21a0c8e0
App 26150 stderr:
App 26150 stderr:   R8: 0x00007f3d1c2bc068
App 26150 stderr:   R9: 0x00007f3d1ddd5b20
App 26150 stderr:  R10: 0x0000000000000001
App 26150 stderr:
App 26150 stderr:  R11: 0x00007f3d14a0e199
App 26150 stderr:  R12: 0x00007f3d21a2f7a0
App 26150 stderr:  R13: 0x00007f3d232ac4d0
App 26150 stderr:
App 26150 stderr:  R14: 0x00007f3d1dd06248
App 26150 stderr:  R15: 0x00007f3d21a2f7a0
App 26150 stderr:  EFL: 0x0000000000010287
App 26150 stderr:
App 26150 stderr:

Guida per i contributori

Apri la guida per i contributori

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Direzione di ricerca

Start with lib/concurrent-ruby/concurrent/atomic/ruby_thread_local_var.rb at line 113 and review the finalization path implicated by the backtrace. Compare the 1.1.6 behavior with 1.1.5 using Ruby 2.5.0 and the reported Passenger setup, then establish a reproducible check for the worker hang or segmentation fault. Done means the failure no longer occurs without regressing thread-local behavior.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
ruby
Ambito
backend
Tipo di issue
Bug
Difficoltà
4/5
Tempo stimato
3-5 giorni
Stato di attività
Ferma
Chiarezza
Da chiarire
Idoneità per principianti
25/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.