Round-robin HA controller can lead to concurrent modications errors
- Dominant language
- Scala
- Stars
- 6.8k
- Forks
- 1.2k
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 2
Description
Since Nginx does proxy in Round-Robin way, some tests which were designed to run sequentially in one controller are failed.
Let me explain one example, travis of latest commit(0737826c06c73a904c49977910d103d1eed79174) failed the test "Wsk Package should allow updating a package". Because the result of package update was 409, not 200
See the code below
```
it should "allow updating a package" in withAssetCleaner(wskprops) { (wp, assetHelper) =>
val name = "simplepackagetoupdate"
assetHelper.withCleaner(wsk.pkg, name) { (pkg, _) =>
pkg.create(name, params1)
pkg.create(name, params2, update = true)
}
}
```
It might runs in this way
```
1. delete pkg when withCleaner starts -> controller0 delete package if it exists
2. pkg.create(name, params1) -> controller1 put package with rev A
3. pkg.create(name, params1) -> controller0 put package with rev B
4. delete pkg when withCleaner ends -> controller1 del package with rev A
```
Package is finally stored with rev B, but controller1 try to delete it with rev A, and then CouchDB response last operation as a conflict(409)
It could be success, if 'cacheInvalidation' msg arrived between 3 & 4. But we can't guarantee that.
I found that many tests were failed by similar scenario while running Travis.
Any good idea?
[tarvis log](https://travis-ci.org/apache/incubator-openwhisk/builds/309179905?utm_source=github_status&utm_medium=notification)
[controller0_logs.log](https://github.com/apache/incubator-openwhisk/files/1516179/controller0_logs.log)
[controller1_logs.log](https://github.com/apache/incubator-openwhisk/files/1516250/controller1_logs.log)
Contributor guide
Assessment
This issue has not been assessed yet.