automl / automl/ConfigSpace

Activity resolution disagrees between grouped and per-child conditions: deactivate_inactive_hyperparameters rejects a configuration check_valid_configuration accepts

Open
#430 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
225
Forks
94
PR merge metrics
No merged PRs in 30d

Description

## Summary

On ConfigSpace 1.2.2 there are two related activity-resolution rules in play, and they disagree for a space where two conditional hyperparameters carry conjunctions built from the same component conditions but of different kinds (`AndConjunction` vs `OrConjunction`):

- `check_configuration` (reached from `Configuration.__init__` → `check_valid_configuration`) and `sample_configuration` both resolve activity from the **grouped** condition nodes in `ConfigurationSpace._dag.minimum_conditions`.
- `deactivate_inactive_hyperparameters` and the public `ConfigurationSpace.parent_conditions_of` mapping resolve it from **each child's own** condition.

The consequences are (1) an internal inconsistency — `deactivate_inactive_hyperparameters` rejects a configuration the library itself accepts and samples — and (2) no public API that reflects the rule the library actually applies.

## Reproduction

Self-contained, against ConfigSpace 1.2.2:

```python
from ConfigSpace import (
AndConjunction, Categorical, Configuration, ConfigurationSpace,
GreaterThanCondition, InCondition, Integer, OrConjunction,
)
from ConfigSpace.util import deactivate_inactive_hyperparameters

space = ConfigurationSpace(name="repro", space={
"family": Categorical("family", ["gbm", "rf", "svm"]),
"depth": Integer("depth", (1, 8), default=6),
"a_shrink": Categorical("a_shrink", ["on", "off"]),
"b_penalty": Categorical("b_penalty", ["on", "off"]),
})
space.add(OrConjunction(
InCondition(space["a_shrink"], space["family"], ["gbm", "rf"]),
GreaterThanCondition(space["a_shrink"], space["depth"], 4),
))
space.add(AndConjunction(
InCondition(space["b_penalty"], space["family"], ["gbm", "rf"]),
GreaterThanCondition(space["b_penalty"], space["depth"], 4),
))

values = {"family": "svm", "depth": 8, "a_shrink": "on", "b_penalty": "on"}

# Accepted here:
Configuration(space, values=values).check_valid_configuration() # OK

# And produced routinely by the space's own sampler:
drawn = [dict(space.sample_configuration()) for _ in range(200)]
print(sum("a_shrink" in d and "b_penalty" in d for d in drawn)) # 165/200

# But rejected here:
deactivate_inactive_hyperparameters(values, space)
# ActiveHyperparameterNotSetError: Hyperparameter is active but has no value set.
```

The cause is visible in the DAG: the two children are grouped under a single node, and the representative is the `OrConjunction`, so `check_configuration` judges **both** children by the `Or` — while `parent_conditions_of["b_penalty"]` still reports the `AndConjunction`.

```python
node = space._dag.minimum_conditions[0]
[space.at[i] for i in node.children_indices] # ['a_shrink', 'b_penalty']
type(node.condition).__name__ # 'OrConjunction'
[type(c).__name__ for c in space.parent_conditions_of["b_penalty"]] # ['AndConjunction']
```

`Conjunction.equivalent_condition_on_parent` groups on matching component parents and values without distinguishing the conjunction kind, which is what brings the two children under one representative. Its own comment notes the grouping is approximate.

## Two things this suggests

**1. `deactivate_inactive_hyperparameters` looks like a genuine bug.** It deactivates using `parent_conditions_of` and then validates the result via `Configuration`, which applies the grouped rule — so on such a space it raises against a configuration it would itself accept. That is independent of any downstream use case.

**2. A public accessor for the grouped conditions would help.** Any consumer that needs to know which hyperparameters are active *before* building a `Configuration` — for instance when driving an external optimiser and wanting to sample only the active dimensions — has to match the rule `check_configuration` applies. Today the only way to do that is to reach into `ConfigurationSpace._dag.minimum_conditions`, since every public route (`parent_conditions_of`, `get_active_hyperparameters`) uses the per-child rule and therefore disagrees on these spaces.

Something like a public `ConfigurationSpace.activity_conditions` (mapping each conditional hyperparameter to the condition that actually decides it), or a public partial-vector activity query, would let consumers stay on supported API. Happy to open a PR for either if you have a preferred shape.

## Environment

- ConfigSpace: distribution `1.2.2` (note that `ConfigSpace.__version__` reports `"1.2.0"` on this release — the module constant appears to lag the distribution metadata; small separate nit)
- Python 3.13, Linux

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with deactivate_inactive_hyperparameters and trace Configuration.__init__ through check_valid_configuration, then compare those results with ConfigurationSpace._dag.minimum_conditions and parent_conditions_of. Reproduce the AndConjunction/OrConjunction case from the issue and add a regression test. Done means activity resolution is consistent for accepted and sampled configurations, with any public-API scope decided explicitly.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.