influxdata / influxdata/influxdb-java

Immediate query after batch write misses points

Aperta
#584 3 commenti 0 reazioni 0 assegnatari Vedi su GitHub
Lingua principale
Java
Stelle
1.2k
Fork
469
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Descrizione

Querying for points immediately after batch write misses points. Example:

```
@Test
public void test()
{
// Query the count of points
Query query = new Query("select count(disk)from disk", databaseName);

// Points to write
int points = BatchOptions.DEFAULTS.getActions() + 1;

db.enableBatch(BatchOptions.DEFAULTS);

long time = System.currentTimeMillis();
for(int i = 0; i < points; i++)
{
db.write(Point.measurement("disk")
.time(time + i, TimeUnit.MILLISECONDS)
.addField("disk", 1)
.build());
}

// Flush final points to influx
db.disableBatch();

QueryResult result = db.query(query);
assertEquals(points, result.getResults().get(0).getSeries().get(0).getValues().size());
}
```

This unit test fails the assert statement saying "Excepted: 1001, Actual: 1". Manually inspecting the database afterwards shows the count query returning all 1001 points as expected.

Here's my influxdb log showing what's happening (sorted by time descending)

```
Mar 21 20:34:05 mschwore71 influxd[21484]: [httpd] 127.0.0.1 - null [21/Mar/2019:20:34:05 +0000] "POST /write?db=influx_tests_c7ccabbe_7bb7_404a_85b7_4d3a9531fb6a&rp=autogen&precision=n&consistency=one HTTP/1.1" 204 0 "-" "okhttp/3.5.0" ab901640-4c18-11e9-8c02-00505684bea9 42142
Mar 21 20:34:05 mschwore71 influxd[21484]: [httpd] 127.0.0.1 - null [21/Mar/2019:20:34:05 +0000] "GET /query?db=influx_tests_c7ccabbe_7bb7_404a_85b7_4d3a9531fb6a&q=select+count%28disk%29from+disk HTTP/1.1" 200 125 "-" "okhttp/3.5.0" ab8e7edf-4c18-11e9-8c01-00505684bea9 769
Mar 21 20:34:05 mschwore71 influxd[21484]: ts=2019-03-21T20:34:05.268221Z lvl=info msg="Executing query" log_id=0EKIBsll000 service=query query="SELECT count(disk) FROM influx_tests_c7ccabbe_7bb7_404a_85b7_4d3a9531fb6a.autogen.disk"
Mar 21 20:34:05 mschwore71 influxd[21484]: [httpd] 127.0.0.1 - null [21/Mar/2019:20:34:05 +0000] "POST /write?db=influx_tests_c7ccabbe_7bb7_404a_85b7_4d3a9531fb6a&rp=autogen&precision=n&consistency=one HTTP/1.1" 204 0 "-" "okhttp/3.5.0" ab8b607d-4c18-11e9-8c00-00505684bea9 13274
```

Stepping through the debugger, I see something like this happening (on my machine at least)

1. [Main Thread] Write buffer is filled to 1k points
2. [Main Thread] Schedule worker thread to write 1k points
3. [Main Thread] Another 1 point is added to the write buffer
4. [Main Thread] Disable batch
5. [Main Thread] Write remaining 1 point in buffer
6. [Main Thread] Query for points
7. [Worker Thread] Write 1k points

Also to note, this causes influxdb-java to write points out of order to influxdb.

Guida per i contributori

Nessuna guida per i contributori indicizzata per questo repository

Direzione di ricerca

Inizia riproducendo lo scenario di unit-test fornito con BatchOptions.DEFAULTS, quindi analizza il batch worker e il comportamento di disableBatch coinvolti nel flush dei punti. Verifica che la disabilitazione del batching attenda la scrittura pianificata di 1.000 punti prima dell'esecuzione della query immediata e che la query restituisca tutti i 1.001 punti senza scritture fuori ordine.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
java
Ambito
databases
Tipo di issue
Bug
Difficoltà
3/5
Tempo stimato
1-2 giorni
Stato di attività
Ferma
Chiarezza
Abbastanza chiara
Idoneità per principianti
35/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.