High memory overhead when patching aws-sdk
- Dominant language
- Ruby
- Stars
- 60
- Forks
- 54
- PR merge metrics
- No merged PRs in 30d
Description
Hello,
We recently added `aws-xray-sdk` to a bunch of applications, and those where we patch `aws_sdk` started using significantly more memory than usual. Here is how it's configured:
```ruby
# if aws-sdk is loaded, we want to instrument that too
patch = Gem.loaded_specs.has_key?('aws-sdk-core') ?
%I[aws_sdk net_http] : %I[net_http]
# if there isn't a name set, attempting to record a segment will
# throw an error
govuk_app_name = ENV['GOVUK_APP_NAME']
name = govuk_app_name.blank? ? 'xray' : govuk_app_name
XRay.recorder.configure(
name: name,
patch: patch,
context_missing: 'LOG_ERROR',
sampling_rules: {
version: 1,
default: {
'fixed_target': ENV.fetch('XRAY_SAMPLE_TARGET', 0).to_i,
'rate': ENV.fetch('XRAY_SAMPLE_RATE', 0.01).to_f,
},
rules: [],
},
)
```
I don't think this is a memory leak, but an overhead incurred when the patching is performed. Here is the memory usage of our publishing-api application, which shot up dramatically when the X-Ray configuration was deployed:

The application doesn't actually use any AWS stuff, but it depends on some AWS gems because there are a couple of rake tasks to import and export data to S3. So despite the application itself not doing any AWS stuff, the memory usage is still dramatically increased.
The reason I think this is a problem with the `aws_sdk` patching is because the same problem does not occur when only `net_http` is patched. The same application, with only `net_http` patched, looks like this:

When the change was deployed, the memory usage plummeted. Towards the end of the graph, a branch which enabled patching of `aws-sdk` was deployed, and the memory usage goes up. This seems like too much of a coincidence for it not to be the `aws-sdk` patching at fault.
There's some more details, including graphs for a second application, here: https://github.com/alphagov/govuk_app_config/pull/61
Contributor guide
Assessment
This issue has not been assessed yet.