Report posterior probability of each proposed axiom in a solution
- Dominant language
- Scala
- Stars
- 37
- Forks
- 2
- PR merge metrics
- No merged PRs in 30d
Description
currently we report the posterior probability of the solution, and the prior probability of each axiom
we should report the posterior of each axiom. We do this by taking the sum of all solutions that include that axiom, and dividing that by the sum of probabilities of solutions that have a different interpretation.
If not all solutions are explored this will be an estimate. The estimate will be biased in favor of those with higher probabilities, as lower priorities may never be explored. We can account for this with a prior probability that the solution is biased
`Pr(Axiom) = IsBiasedPrior * AxiomPrior + (1-IsBiasedPrior) * AxiomEstimatedPosterior`
We can estimate IsBiasedPrior based on how many times the axiom or its alternatives were explored in the overall search tree.
We can also have strategies to minimize bias. E.g for each potential axiom we start at least one search with that as our initial choice.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by locating the code that reports the solution posterior and each axiom's prior, then trace how explored solutions and alternatives are represented in the search tree. Review the issue's proposed aggregation and bias-estimation approach before deciding on the design. Done means the reporting includes a posterior estimate for every proposed axiom, including cases where not all solutions are explored.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- scala
- Domain
- backend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100