kragniz / kragniz/python-etcd3
watch key is CPU100 dead
- Lingua principale
- Python
- Stelle
- 450
- Fork
- 194
- Metriche di merge delle PR
- Nessuna PR unita negli ultimi 30g
Descrizione
centos7.2, python2.7 etcd3 0.5.2
I put 20w key to etcd3, try to test python-etcd3 watch performance.
```python
def watch_node():
host = random.choice(ETCD_SERVER)
etcd = etcd3.client(host=host[0], port=host[1])
watch_count = 0
events_iterator, cancel = etcd.watch_prefix('/')
f = open('etcdout', 'w')
for event in events_iterator:
# print(watch_count, event)
f.write(str(watch_count))
f.write("\n")
f.flush()
# f.write(str(event))
watch_count += 1
if watch_count > 1000000:
cancel()
```
```go
package main
import "fmt"
import "time"
import (
"log"
"math/rand"
"os"
"golang.org/x/net/context"
"github.com/coreos/etcd/clientv3"
)
func worker(id int, jobs <-chan int, results chan<- int) {
num := rand.Int31n(10)
time.Sleep(time.Duration(num) * time.Second)
cfg := clientv3.Config{
Endpoints: []string{"http://10.10.3.10:2379", "http://10.1.106.39:2379", "http://10.15.12.253:2379"},
DialTimeout: 5 * time.Second,
}
cli, err := clientv3.New(cfg)
if err != nil {
log.Fatal(err)
}
for j := range jobs {
// ctx, cancel := context.WithTimeout(context.Background(), 3)
resp, err := cli.Put(context.Background(), fmt.Sprintf("/computed/host_vm/%i/%i/state", id, j), "RUNNING")
// defer cancel()
log.Println("xxxxxxxxxxxxxxxxxxxxxx")
if err != nil {
log.Println(err)
cli, err = clientv3.New(cfg)
if err != nil {
log.Println(err)
}
} else {
// print common key info
_ = resp
log.Printf("Set is done. Metadata is %q\n", resp)
}
time.Sleep(10 * time.Second)
results <- j
}
// for j := range jobs {
// fmt.Println("worker", id, "started job", j)
// time.Sleep(time.Second)
// fmt.Println("worker", id, "finished job", j)
// results <- j * 2
// }
}
func main() {
f, err := os.OpenFile("testlogfile", os.O_RDWR | os.O_CREATE | os.O_APPEND, 0666)
if err != nil {
fmt.Println("error opening file: %v", err)
}
defer f.Close()
log.SetOutput(f)
jobs := make(chan int, 500000)
results := make(chan int, 500000)
for w := 1; w <= 30000; w++ {
go worker(w, jobs, results)
}
for j := 1; j <= 500000; j++ {
jobs <- j
}
close(jobs)
for a := 1; a <= 500000; a++ {
<-results
}
}
```
```
(gdb) bt
#0 0x00007f3ab2d4e790 in sem_wait () from /lib64/libpthread.so.0
#1 0x00007f3ab30697b5 in PyThread_acquire_lock () from /lib64/libpython2.7.so.1.0
#2 0x00007f3ab306d442 in lock_PyThread_acquire_lock () from /lib64/libpython2.7.so.1.0
#3 0x00007f3ab303daa4 in PyEval_EvalFrameEx () from /lib64/libpython2.7.so.1.0
#4 0x00007f3ab303f0bd in PyEval_EvalCodeEx () from /lib64/libpython2.7.so.1.0
#5 0x00007f3ab303d76f in PyEval_EvalFrameEx () from /lib64/libpython2.7.so.1.0
#6 0x00007f3ab303f0bd in PyEval_EvalCodeEx () from /lib64/libpython2.7.so.1.0
#7 0x00007f3ab303d76f in PyEval_EvalFrameEx () from /lib64/libpython2.7.so.1.0
#8 0x00007f3ab2fc1908 in gen_send_ex.isra.0 () from /lib64/libpython2.7.so.1.0
#9 0x00007f3ab303b4bd in PyEval_EvalFrameEx () from /lib64/libpython2.7.so.1.0
```
Guida per i contributori
Apri la guida per i contributori
Direzione di ricerca
Iniziate dalla chiamata watch_prefix('/') nel riproduttore Python fornito e confrontatene il comportamento con il workload Go associato e il backtrace di gdb. Riproducete il watch bloccato con le versioni indicate di Python, etcd3 e CentOS, quindi individuate e risolvete la causa del blocco della CPU; il lavoro è completato quando il watch elabora il workload oppure fallisce con un errore chiaro e riproducibile.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Valutazione
- Stack tecnologico
- python
- Ambito
- api, backend
- Tipo di issue
- Bug
- Difficoltà
- 4/5
- Tempo stimato
- 3-5 giorni
- Stato di attività
- Ferma
- Chiarezza
- Da chiarire
- Idoneità per principianti
- 25/100