ruby-concurrency / ruby-concurrency/concurrent-ruby

Memory leak in Concurrent::Promises.future

Open
#960 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Ruby
Stars
5.8k
Forks
420
Avg merge
20h 45m
Merged PRs (30d)
4

Description

* Operating system:                linux
* Ruby implementation:             Ruby
* `concurrent-ruby` version:       1.1.10
* `concurrent-ruby-ext` installed: no
* `concurrent-ruby-edge` used:     no

The test script

# frozen_string_literal: true

require 'bundler/inline'

gemfile(true) do
  source 'https://rubygems.org'

  gem 'concurrent-ruby', '1.1.10'
  gem 'memory_profiler', '~> 1'
end

class Thing; end

def report(title, &block)
  puts title

  pp MemoryProfiler.report(&block).retained_memory_by_class
end

report('Warmup') do
  Concurrent::Promises.future { Thing.new }.wait
end

report('When waiting for the Future') do
  Concurrent::Promises.future { Thing.new }.wait
end

report('When waiting for the Future with args') do
  Concurrent::Promises.future(Thing.new) { |o| o }.wait
end

report('When waiting for the Future and actively dereferencing it') do
  x = Concurrent::Promises.future { Thing.new }.wait
  x = nil
end

report('When waiting for the Future with args and actively dereferencing it') do
  x = Concurrent::Promises.future(Thing.new) { |o| o }.wait
  x = nil
end

yields the following output

Fetching gem metadata from https://rubygems.org/..
Resolving dependencies...
Using bundler 2.3.19
Using memory_profiler 1.0.0
Using concurrent-ruby 1.1.10
Warmup
[{:data=>"Thread", :count=>1048992},
 {:data=>"Concurrent::CachedThreadPool", :count=>216},
 {:data=>"Thread::Mutex", :count=>216},
 {:data=>"Array", :count=>200},
 {:data=>"Thread::ConditionVariable", :count=>192},
 {:data=>"Concurrent::Event", :count=>144},
 {:data=>"Proc", :count=>80},
 {:data=>"String", :count=>80},
 {:data=>"Thread::Queue", :count=>76},
 {:data=>"Concurrent::RubyThreadPoolExecutor::Worker", :count=>40}]
When waiting for the Future
[{:data=>"Array", :count=>40}]
When waiting for the Future with args
[{:data=>"Array", :count=>40}]
When waiting for the Future and actively dereferencing it
[{:data=>"Array", :count=>40}]
When waiting for the Future with args and actively dereferencing it
[{:data=>"Array", :count=>40}]

Note that in contrast to #959 the return value of the block is not leaking here.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at Concurrent::Promises.future and reproduce the report with the supplied Ruby script and concurrent-ruby 1.1.10. Trace retained objects around each future and wait call; done means the reported memory retention is resolved without changing the non-leaking return-value behavior noted in the issue.

Written by the indexing model from the issue text.

Assessment

Tech stack
ruby
Domain
backend, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.