Allow setting raise_on_mismatches to base class level for tests
- Dominant language
- Ruby
- Stars
- 7.8k
- Forks
- 505
- PR merge metrics
- No merged PRs in 30d
Description
We are using a base experiment class to manage our publish logic. All the actual experiments are extending from this base class. However, if we set raise_on_mismatches to this base class the result doesn't take effect as the method is using class instance variables(instead of class variables). In the end, people occasionally forget to use this test helper for their newly created experiments.
It would be great if we can manage this test helper via a base class. To elaborate more on this, this is somewhat our structure;
```ruby
class BaseExperiment
include Scientist::Experiment
def initialize
# setting up some instance variables mainly needed for publish logic
end
def publish(result)
# some custom logic
end
end
class WidgetExperiment < BaseExperiment
def initialize
# setting up custom variables
super
end
def enabled?
# custom enabled logic
end
end
class AnotherExperiment < BaseExperiment
# similar context with widget experiment
end
```
What we want is to use `BaseExperiment.raise_on_mismatches = true` to ensure all child experiments are tested on tests without needed a separate test setup. Would such a need make sense to you?
Contributor guide
Research direction
Start by tracing the raise_on_mismatches implementation in Scientist::Experiment and how it is configured on experiment classes. Add coverage for setting BaseExperiment.raise_on_mismatches and verify that child experiments honor the setting without separate test setup.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- ruby
- Domain
- backend
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100