couchbaselabs / couchbaselabs/sgload
Panic calling pullMoreDocs Expected 100 docs, got 46 docs
- Dominant language
- Go
- Stars
- 6
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
To reproduce:
```
$ python libraries/provision/create_and_instantiate_cluster.py --stackname="TleydenPerf" --num-servers=3 --server-type="m4.2xlarge" --num-sync-gateways=4 --sync-gateway-type="m4.large" --num-gatlings=2 --gatling-type="m3.medium" --num-lbs=0
$ python libraries/provision/provision_cluster.py --server-version=4.1.0 --sync-gateway-version=1.4-19 --sync-gateway-config-file=resources/sync_gateway_configs/performance/sync_gateway_default_performance.json --install-deps
$ go run main.go gateload --createreaders --createwriters
--numreaders 5000 --numwriters 5000 --numupdaters 0
--numrevsperdoc 2 --numdocs 500000 --numchannels 1
--batchsize 1 --expvarprogressenabled --writerdelayms 10000
--loglevel debug
--sg-url http://s61705cnt72.sc.couchbase.com:4984/db/
```
and seeing this panic:
```
DBUG[01-06|16:42:21] Statsd counter for get_document_counter n=100
EROR[01-06|16:42:21] Error calling pullMoreDocs agent.ID=174 since= err="Expected 100 docs, got 46 docs"
INFO[01-06|16:42:21] Reader finished reader=reader-user-174-3b822651-f361-4c35-89b9-7bedcb79c129
INFO[01-06|16:42:21] Reader finished agent.ID=174 numdocs=0
panic: Error calling pullMoreDocs: Expected 100 docs, got 46 docs
goroutine 20181 [running]:
panic(0x80fb00, 0xc43c46c860)
/usr/local/go/src/runtime/panic.go:500 +0x1a1
github.com/couchbaselabs/sgload/sgload.(*Reader).Run(0xc4228389c0)
/opt/go/src/github.com/couchbaselabs/sgload/sgload/reader.go:130 +0xc82
created by github.com/couchbaselabs/sgload/sgload.GateLoadRunner.startReaders
/opt/go/src/github.com/couchbaselabs/sgload/sgload/gateload_runner.go:228 +0xff
exit status 2
```
Not sure if it's valid for `_bulk_get` to return partial docs like this.
@adamcfraser does this look like correct Sync Gateway behavior that sgload should handle? (assuming there is no bugs in sgload where it doesn't actually request 100 docs and get 46 docs)
Contributor guide
No contributing guide indexed for this repository
Research direction
Start in sgload/reader.go at Reader.Run line 130 and trace pullMoreDocs, then review the _bulk_get response described in the reproduction. Determine whether Sync Gateway may return fewer documents than requested and whether sgload should handle that case. Done means the expected behavior is established and the reader no longer panics unexpectedly when a partial response is returned.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100