ruby-concurrency / ruby-concurrency/concurrent-ruby

Memory leak in Concurrent::Future

Open
#959 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Ruby
Stars
5.8k
Forks
420
Avg merge
20h 45m
Merged PRs (30d)
4

Description

* Operating system:                linux
* Ruby implementation:             MRI 2.6.6, 2.7.4, 3.1.2
* `concurrent-ruby` version:       1.1.10
* `concurrent-ruby-ext` installed: no
* `concurrent-ruby-edge` used:     no

When passing a block to Concurrent::Future.execute and waiting for the result, the Future can be garbage collected, but the return value of the block seems to stay in memory forever, as demonstrated by the following script:

# frozen_string_literal: true

require 'bundler/inline'

gemfile(true) do
  source 'https://rubygems.org'

  gem 'concurrent-ruby', '1.1.10'
end

class Thing; end

def block_while_things_in_memory(title)
  i = 0

  until ObjectSpace.each_object(Thing).count == 0
    i += 1
    puts "#{title} - #{i}. GC - #{ObjectSpace.each_object(Thing).count} Things, #{ObjectSpace.each_object(Concurrent::Future).count} Futures"
    GC.start
  end

  puts "#{title}: No more Things in memory"
end

block_while_things_in_memory('Initial cleanup')

Concurrent::Future.execute do
  Thing.new
  nil
end.wait

block_while_things_in_memory('When block returns nil')

Concurrent::Future.execute do
  Thing.new
end.wait

block_while_things_in_memory('When block returns a Thing')

When running the script on several different versions of MRI, I always get something like this never ending output:

Fetching gem metadata from https://rubygems.org/..
Resolving dependencies...
Using bundler 2.3.7
Using concurrent-ruby 1.1.10
Initial cleanup: No more Things in memory
When block returns nil - 1. GC - 1 Things, 1 Futures
When block returns nil: No more Things in memory
When block returns a Thing - 1. GC - 1 Things, 1 Futures
When block returns a Thing - 2. GC - 1 Things, 0 Futures
When block returns a Thing - 3. GC - 1 Things, 0 Futures
When block returns a Thing - 4. GC - 1 Things, 0 Futures
When block returns a Thing - 5. GC - 1 Things, 0 Futures
When block returns a Thing - 6. GC - 1 Things, 0 Futures
When block returns a Thing - 7. GC - 1 Things, 0 Futures
When block returns a Thing - 8. GC - 1 Things, 0 Futures
When block returns a Thing - 9. GC - 1 Things, 0 Futures
When block returns a Thing - 10. GC - 1 Things, 0 Futures
When block returns a Thing - 11. GC - 1 Things, 0 Futures
When block returns a Thing - 12. GC - 1 Things, 0 Futures
When block returns a Thing - 13. GC - 1 Things, 0 Futures
When block returns a Thing - 14. GC - 1 Things, 0 Futures
When block returns a Thing - 15. GC - 1 Things, 0 Futures
When block returns a Thing - 16. GC - 1 Things, 0 Futures
When block returns a Thing - 17. GC - 1 Things, 0 Futures
When block returns a Thing - 18. GC - 1 Things, 0 Futures
When block returns a Thing - 19. GC - 1 Things, 0 Futures
When block returns a Thing - 20. GC - 1 Things, 0 Futures
When block returns a Thing - 21. GC - 1 Things, 0 Futures
When block returns a Thing - 22. GC - 1 Things, 0 Futures
When block returns a Thing - 23. GC - 1 Things, 0 Futures
When block returns a Thing - 24. GC - 1 Things, 0 Futures
When block returns a Thing - 25. GC - 1 Things, 0 Futures
When block returns a Thing - 26. GC - 1 Things, 0 Futures
When block returns a Thing - 27. GC - 1 Things, 0 Futures
When block returns a Thing - 28. GC - 1 Things, 0 Futures
When block returns a Thing - 29. GC - 1 Things, 0 Futures
...

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the Concurrent::Future.execute and wait calls in the provided reproduction, then trace how completed return values are retained. Run the ObjectSpace script as a baseline; done means the returned Thing becomes collectible after wait rather than remaining in memory indefinitely.

Written by the indexing model from the issue text.

Assessment

Tech stack
ruby
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.