github / github/scientist

Allow setting raise_on_mismatches to base class level for tests

Open
#174 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Ruby
Stars
7.8k
Forks
505
PR merge metrics
No merged PRs in 30d

Description

We are using a base experiment class to manage our publish logic. All the actual experiments are extending from this base class. However, if we set raise_on_mismatches to this base class the result doesn't take effect as the method is using class instance variables(instead of class variables). In the end, people occasionally forget to use this test helper for their newly created experiments.

It would be great if we can manage this test helper via a base class. To elaborate more on this, this is somewhat our structure;

```ruby
class BaseExperiment
include Scientist::Experiment

def initialize
# setting up some instance variables mainly needed for publish logic
end

def publish(result)
# some custom logic
end
end

class WidgetExperiment < BaseExperiment
def initialize
# setting up custom variables
super
end

def enabled?
# custom enabled logic
end
end

class AnotherExperiment < BaseExperiment
# similar context with widget experiment
end
```

What we want is to use `BaseExperiment.raise_on_mismatches = true` to ensure all child experiments are tested on tests without needed a separate test setup. Would such a need make sense to you?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.