ValueError: math domain error in PC with missing data
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 274
- PR merge metrics
- No merged PRs in 30d
Description
Hi there,
my input data is like this:
I then want to discover with missing values:
from causallearn.search.ConstraintBased.PC import pc
dataset= X.to_numpy()
sub_cols = X.columns
# default parameters
cg = pc(dataset,alpha=0.05,indep_test='mv_fisherz',mvpc=True)
Full error:
ValueError Traceback (most recent call last)
Cell In[206], line 5
3 sub_cols = X.columns
4 # default parameters
----> 5 cg = pc(dataset,alpha=0.05,indep_test='mv_fisherz',mvpc=True)
File /opt/conda/lib/python3.10/site-packages/causallearn/search/ConstraintBased/PC.py:41, in pc(data, alpha, indep_test, stable, uc_rule, uc_priority, mvpc, correction_name, background_knowledge, verbose, show_progress, node_names, **kwargs)
39 if indep_test == fisherz:
40 indep_test = mv_fisherz
---> 41 return mvpc_alg(data=data, node_names=node_names, alpha=alpha, indep_test=indep_test, correction_name=correction_name, stable=stable,
42 uc_rule=uc_rule, uc_priority=uc_priority, background_knowledge=background_knowledge,
43 verbose=verbose,
44 show_progress=show_progress, **kwargs)
45 else:
46 return pc_alg(data=data, node_names=node_names, alpha=alpha, indep_test=indep_test, stable=stable, uc_rule=uc_rule,
47 uc_priority=uc_priority, background_knowledge=background_knowledge, verbose=verbose,
48 show_progress=show_progress, **kwargs)
File /opt/conda/lib/python3.10/site-packages/causallearn/search/ConstraintBased/PC.py:200, in mvpc_alg(data, node_names, alpha, indep_test, correction_name, stable, uc_rule, uc_priority, background_knowledge, verbose, show_progress, **kwargs)
198 indep_test = CIT(data, indep_test, **kwargs)
199 ## Step 1: detect the direct causes of missingness indicators
--> 200 prt_m = get_parent_missingness_pairs(data, alpha, indep_test, stable)
201 # print('Finish detecting the parents of missingness indicators. ')
202
203 ## Step 2:
204 ## a) Run PC algorithm with the 1st step skeleton;
205 cg_pre = SkeletonDiscovery.skeleton_discovery(data, alpha, indep_test, stable,
206 background_knowledge=background_knowledge,
207 verbose=verbose, show_progress=show_progress, node_names=node_names)
File /opt/conda/lib/python3.10/site-packages/causallearn/search/ConstraintBased/PC.py:275, in get_parent_missingness_pairs(data, alpha, indep_test, stable)
272 ## Get the index of parents of missingness indicators
273 # If the missingness indicator has no parent, then it will not be collected in prt_m
274 for missingness_i in missingness_index:
--> 275 parent_of_missingness_i = detect_parent(missingness_i, data, alpha, indep_test, stable)
276 if not isempty(parent_of_missingness_i):
277 parent_missingness_pairs['prt'].append(parent_of_missingness_i)
File /opt/conda/lib/python3.10/site-packages/causallearn/search/ConstraintBased/PC.py:363, in detect_parent(r, data_, alpha, indep_test, stable)
361 if len(Neigh_x) >= depth:
362 for S in combinations(Neigh_x, depth):
--> 363 p = cg.ci_test(x, y, S)
364 if p > alpha:
365 if not stable: # Unstable: Remove x---y right away
File /opt/conda/lib/python3.10/site-packages/causallearn/graph/GraphClass.py:58, in CausalGraph.ci_test(self, i, j, S)
56 # assert i != j and not i in S and not j in S
57 if self.test.method == 'mc_fisherz': return self.test(i, j, S, self.nx_skel, self.prt_m)
---> 58 return self.test(i, j, S)
File /opt/conda/lib/python3.10/site-packages/causallearn/utils/cit.py:388, in MV_FisherZ.__call__(self, X, Y, condition_set)
386 if abs(r) >= 1: r = (1. - np.finfo(float).eps) * np.sign(r) # may happen when samplesize is very small or relation is deterministic
387 Z = 0.5 * log((1 + r) / (1 - r))
--> 388 X = sqrt(len(test_wise_deletion_XYcond_rows_index) - len(condition_set) - 3) * abs(Z)
389 p = 2 * (1 - norm.cdf(abs(X)))
390 self.pvalue_cache[cache_key] = p
ValueError: math domain error
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the call in search/ConstraintBased/PC.py with the reported missing-data configuration, then trace the failing calculation in utils/cit.py, especially MV_FisherZ.call. Check how few valid test-wise rows and conditioning-set size are handled, and add a regression test that defines the expected behavior instead of raising ValueError.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 32/100