Echtzeitsysteme / Echtzeitsysteme/java-refactoring-ttc
Execution time measurements
- Dominant language
- Java
- Stars
- 0
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
The case description demands solution authors to put the execution times of all test cases as measured by ARTE into the solution descriptions. However, all test cases are very small toy examples. When I run `execute --all` the worst execution time I get for my solution is ~0.7 seconds for `pub_pum3_1`. However, when I execute just that using `execute --test pub_pum3_1` it finishes in ~0.02 which is about the time needed for the other test cases, too. So that single longer execution time seems to be some GC hiccup which only happens when executing all tests in a sequence or something alike.
Well, long story short: I doubt the execution times measured for these toy examples is significant. Can't you provide some performance test cases, e.g., a base case with 10 classes with 5 pullable and 5 non-pullable members each, and then further cases where the base case is duplicated several times to get some larger models? (Like with the train benchmark case where the models double in size.)
Also, I think as soon as you use something like JaMoPP or Modisco, the execution times are probably dominated by parsing, reference resolution, and serialization, i.e., all the stuff which has actually nothing to do with the actual refactorings. So an execution sequence like `parseJava -> transformToPG -> 1000 times: refactor -> serializeAsJava` would be more meaningful with respect to execution time than the sequences containing at most two refactorings which ARTE does right now.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.