ASSERT-KTH / ASSERT-KTH/sequencer
improve beam search
- Dominant language
- Java
- Stars
- 87
- Forks
- 40
- PR merge metrics
- No merged PRs in 30d
Description
right now, we're using beam search as an off-the-shelf component.
It would be great that:
* the search embeds some kind of patch quality knowledge: the first patch generated should have a better quality than
* the search ensures some kind of diversity in the way the bug is being fixed
Ideas from UCDavis:
- by raising the "temperature" (AFAIU adding a multiplicative factor on the softmax)
- by using random sampling instead of beam search
Related work:
- [Diverse beam search: Decoding diverse solutions from neural sequence models](https://arxiv.org/pdf/1610.02424)
- [Improved Beam Search Diversity for Neural MachineTranslation with k-DPP Sampling](http://web.stanford.edu/class/cs224n/reports/custom/15709706.pdf)
> By sampling text from the dynamic nucleus of the probability distribution, which allows for diversity while effectively truncating the less reliable tail of the distribution, the resulting text better demonstrates the quality of human text, yielding enhanced diversity without sacrificing fluency and coherence.
[The Curious Case of Neural Text Degeneration](https://arxiv.org/pdf/1904.09751.pdf)
Contributor guide
No contributing guide indexed for this repository
Research direction
The issue does not name implementation files, tests, or an entry point. Locate the current beam-search integration, then read the cited work on diverse beam search, sampling, and neural text degeneration. The work is done when patch quality improves while generated fixes show meaningful diversity, with evaluation criteria agreed before implementation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100