Negative shapely values in tunability()?
- Dominant language
- Python
- Stars
- 21
- Forks
- 5
- PR merge metrics
- No merged PRs in 30d
Description
What do negative shapely values mean? Is this a bug?
The way I understood it, they should not be possible.
## Minimal reproduction
```python
import numpy as np
from ConfigSpace import ConfigurationSpace, Integer, Configuration
from hypershap import HyperSHAP, ExplanationTask
rng = np.random.RandomState(4)
cs = ConfigurationSpace(seed=4)
for i in range(15):
cs.add([Integer(f"x{i}", (0, 29), default=15)])
data = []
for _ in range(1000):
vals = {f"x{i}": rng.randint(0, 30) for i in range(15)}
c = Configuration(cs, vals)
cost = sum(vals.values()) + rng.normal(0, 5)
data.append((c, cost))
hshap = HyperSHAP(ExplanationTask.from_data(cs, data), approximation_budget=2**9)
iv = hshap.tunability(order=1, baseline_config=cs.get_default_configuration(), index="FSII")
neg = [(k, v) for k, v in iv.dict_values.items() if k and v < 0]
print(f"Negative values: {neg}")
```
Output:
```
Negative values: [((1,), -1.001706630271433)]
```
Reproduced with:
- `hypershap==0.0.6`
- `shapiq==1.4.1`
- `ConfigSpace==1.2.2`
- `scikit-learn==1.8.0rc1`
- `numpy==2.2.6`
## Possible issues
1. hypershap/utils.py:115 declares `self.colation_cache = {}` but this isn't actually used anywhere. The comment above says: `# cache coalition values to ensure monotonicity for min/max`. Maybe that means monotonicity is not ensured?
2. This issue seemingly appears with many hyperparameters, so maybe it's an issue of the approximator that kicks in with more than 14 HP. Can the approximation produce negative values? If yes maybe clip the approximated negative SI values to 0.
Contributor guide
Research direction
Start with the minimal reproduction for HyperSHAP.tunability(order=1, index="FSII") and inspect hypershap/utils.py around line 115, including the unused colation_cache. Trace how coalition values and approximated interaction values are computed for the 15-parameter case. Done means identifying whether the negative value is expected or a bug and adding or updating coverage for the reported behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100