Trusted-AI / Trusted-AI/AIX360
Questions about the results obtained by XAI method
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.8k
- Forks
- 327
- Avg merge
- 1h 13m
- Merged PRs (30d)
- 1
Description
I found a strange phenomenon. For the same model, the same training sample and test sample, other operations are identical. Theoretically, the values obtained by using the XAI method (like Saliency) to evaluate the interpretability of the model should be the same. However, I retrained a new model, and the interpretability values obtained are completely different from those obtained from the previous model. Does anyone know why this happens? The interpretability value is completely unstable, and the results cannot be reproduced. Unless I completely save this model after training it, and then reload this parameter, the results will be the same. Does anyone know why
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing how the model is retrained and how the Saliency-based interpretability value is computed. Compare repeated runs with identical training and test samples, including the saved-and-reloaded model case. Done means identifying the source of the differing results or documenting the required conditions for reproducibility.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100