elastic / elastic/logstash

task: detangle build-time and runtime dependencies

Open
#17,517 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
14.9k
Forks
3.5k
Avg merge
19h 14m
Merged PRs (30d)
63

Description

There are a few interconnected issues with how we use bundler for both the build chain and the plugin manager. This is not an exhaustive list.

- The runtime and development dependencies of logstash-core are somewhat arbitrarily split between `Gemfile.template` and `logstash-core/logstash-core.gemspec`
- use `gemspec path: 'logstash-core'` in the `Gemfile.template` instead of making `logstash-core` a runtime dependency; this ensures that the `add_development_dependency` declarations in `logstash-core/logstash-core.gemspec` are effectively development dependencies of the `Gemfile.template`
- move non-plugin (a) runtime dependency declarations and (b) development declarations from `Gemfile.template` to the gemspec
- NOTE: version constraints for plugins must be in `Gemfile.template` in order to support plugin upgrades in shipped artifacts
- temporary pins of _transitive_ dependencies can continue to live in `Gemfile.template`
- `bin/logstash-plugin install --development` installs not only the development dependencies of logstash core, but also of _each of the plugins that are currently installed_, and it does not _merge_ the requirements so that the last one wins (even if it overrides a _runtime_ requirement that is in the checked-in `logstash-core.gemspec` or `Gemfile.template`).
- do we ever _actually_ need the development dependencies of all of the plugins while developing Logstash core? Why aren't the dev dependencies we need specified as development dependencies of logstash core?
- Because we build artifacts directly from source, the dependencies that are required for building the artifacts but _not_ required for the built artifact itself are a part of the shipped bundle (e.g., `rake`, `octokit`, `childprocess`, `minitar`, `fpm`).
- I _believe_ this can be solved by using an intermediate staging directory and installing its bundle using its plugin manager, at which point the `build` group of dependencies in the `Gemfile.template` could be made development dependencies in `logstash-core/logstash-core.gemspec`, which would ensure that they're never installed in the built artifact.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.