ASSERT-KTH / ASSERT-KTH/sequencer

improve beam search

Open
#16 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
87
Forks
40
PR merge metrics
No merged PRs in 30d

Description

right now, we're using beam search as an off-the-shelf component.

It would be great that:

* the search embeds some kind of patch quality knowledge: the first patch generated should have a better quality than
* the search ensures some kind of diversity in the way the bug is being fixed

Ideas from UCDavis:

- by raising the "temperature" (AFAIU adding a multiplicative factor on the softmax)
- by using random sampling instead of beam search

Related work:

- [Diverse beam search: Decoding diverse solutions from neural sequence models](https://arxiv.org/pdf/1610.02424)
- [Improved Beam Search Diversity for Neural MachineTranslation with k-DPP Sampling](http://web.stanford.edu/class/cs224n/reports/custom/15709706.pdf)

> By sampling text from the dynamic nucleus of the probability distribution, which allows for diversity while effectively truncating the less reliable tail of the distribution, the resulting text better demonstrates the quality of human text, yielding enhanced diversity without sacrificing fluency and coherence.

[The Curious Case of Neural Text Degeneration](https://arxiv.org/pdf/1904.09751.pdf)

Contributor guide

No contributing guide indexed for this repository

Research direction

The issue does not name implementation files, tests, or an entry point. Locate the current beam-search integration, then read the cited work on diverse beam search, sampling, and neural text degeneration. The work is done when patch quality improves while generated fixes show meaningful diversity, with evaluation criteria agreed before implementation.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.