ASSERT-KTH / ASSERT-KTH/spork

DIfferent bugs resulting in invalid merge outputs found during an experiment

Open
#533 11 comments 4 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
72
Forks
11
PR merge metrics
No merged PRs in 30d

Description

Hi, I'm part of a research paper evaluating various merge algorithms. We noticed that while Spork performed exceptionally well, it also sometimes made unexpected and invalid outputs. We'd like to share some of these patterns with you in hopes that you may be familiar with them and know what could be causing them (we suspect Spoon and the pretty-printing process).

Our testing framework is hosted at https://github.com/benedikt-schesch/AST-Merging-Evaluation/. There are instructions in the README that explain how to reproduce our results. Specifically, at the bottom are instructions on how to replay a merge provided an ID so you may view the full context of the merge. IDs are provided in parentheses below.

This is a long list of examples and we'd appreciate any insight as to potential causes for them.

# Non-compilable errors
These are merge outputs that produce invalid Java code.

* [ ] Places Parentheses Incorrectly (345-203)
Expected:
```java
Assertions.assertThat(ex.getMessage().contains(expectedError));
^
```
Spork:
```java
Assertions.assertThat(ex.getMessage()).contains(expectedError);
^
```

* [ ] Classes With the Same Name (1322-24)
This is an imported name conflict.
Expected:
```java
protected javax.jms.Message convertToJMSMessage ...
^^^^^^^^^^
```
Spork:
```java
protected Message convertToJMSMessage ...
^
```

This conflicted with another package that exported a class named `Message`. It was previously disambiguated by using `javax.jms.Message`.

* [ ] Omission of Types (1741-4)
Expected:
```java
} catch (Exception e) {
^^^^^^^^^
```
Spork:
```java
} catch ( e) {
^
```

* [ ] Incorrect Placement of Generics (2955-13)
Expected:
```java
new UnsignedVariableBitLengthType( img, nBits ) );
^^^^^
```
Spork:
```java
new UnsignedVariableBitLengthType(img));
^^^^^^^
```

* [ ] Dropping Escape Characters (4595-12)
Expected
```java
return "\"";
^
```
Spork:
```java
return """;
^
```

* [ ] Merge Cascaded Conditionals (1885-445)
Expected:
```java
} else {
if (!daemon) {
log.info("K3PO started (CTRL+C to stop)");
}
else {
log.info("K3PO started");
}
}
```
Spork:
```java
} else if (!daemon) {
log.info("K3PO started (CTRL+C to stop)");
} else {
log.info("K3PO started");
}
```

While this one did not produce a compilation error in this particular example, it is a risky transformation.

# Compilable errors
These were outputs that still produced valid Java code, but were unexpected.

* [ ] Adds Parentheses (2995-13)
This one is particularly common in our findings and we wonder if it's related to the pretty printer.

Expected:
```java
this( ( NativeImg< ?, ? extends LongAccess > ) null )
```
Spork:
```java
this(((NativeImg) (null)));
^ ^ ^^
```

* [ ] Added Extra Semicolons (1741-4)
```java
int to = (int) docTrees.getSourcePositions().getEndPosition(pkgTree.
getCompilationUnit(), doc, node);;
^
```
```java
SearchStrategyModule stratModule = new SearchStrategyModule() {
[...]
};;;
^^
```

* [ ] Swaps Qualifier Positions (4959-12)
Expected:
```java
private final static Map RULE_MAP = new HashMap();
^^^^^ ^^^^^^
```
Spork
```java
private static final Map RULE_MAP = new HashMap();
^^^^^^ ^^^^^
```

Thank you for your review!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the external AST-Merging-Evaluation README and its instructions for replaying merge IDs, then inspect the listed examples such as 345-203 and 2995-13. Compare Spork's outputs with the expected Java for each case and investigate the suspected Spoon and pretty-printing stages. Done would require identifying and addressing the causes across the reported invalid and unexpected outputs.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
compilers, tooling
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.