influxdata / influxdata/influxdb-java
Immediate query after batch write misses points
- Lingua principale
- Java
- Stelle
- 1.2k
- Fork
- 469
- Metriche di merge delle PR
- Nessuna PR unita negli ultimi 30g
Descrizione
Querying for points immediately after batch write misses points. Example:
```
@Test
public void test()
{
// Query the count of points
Query query = new Query("select count(disk)from disk", databaseName);
// Points to write
int points = BatchOptions.DEFAULTS.getActions() + 1;
db.enableBatch(BatchOptions.DEFAULTS);
long time = System.currentTimeMillis();
for(int i = 0; i < points; i++)
{
db.write(Point.measurement("disk")
.time(time + i, TimeUnit.MILLISECONDS)
.addField("disk", 1)
.build());
}
// Flush final points to influx
db.disableBatch();
QueryResult result = db.query(query);
assertEquals(points, result.getResults().get(0).getSeries().get(0).getValues().size());
}
```
This unit test fails the assert statement saying "Excepted: 1001, Actual: 1". Manually inspecting the database afterwards shows the count query returning all 1001 points as expected.
Here's my influxdb log showing what's happening (sorted by time descending)
```
Mar 21 20:34:05 mschwore71 influxd[21484]: [httpd] 127.0.0.1 - null [21/Mar/2019:20:34:05 +0000] "POST /write?db=influx_tests_c7ccabbe_7bb7_404a_85b7_4d3a9531fb6a&rp=autogen&precision=n&consistency=one HTTP/1.1" 204 0 "-" "okhttp/3.5.0" ab901640-4c18-11e9-8c02-00505684bea9 42142
Mar 21 20:34:05 mschwore71 influxd[21484]: [httpd] 127.0.0.1 - null [21/Mar/2019:20:34:05 +0000] "GET /query?db=influx_tests_c7ccabbe_7bb7_404a_85b7_4d3a9531fb6a&q=select+count%28disk%29from+disk HTTP/1.1" 200 125 "-" "okhttp/3.5.0" ab8e7edf-4c18-11e9-8c01-00505684bea9 769
Mar 21 20:34:05 mschwore71 influxd[21484]: ts=2019-03-21T20:34:05.268221Z lvl=info msg="Executing query" log_id=0EKIBsll000 service=query query="SELECT count(disk) FROM influx_tests_c7ccabbe_7bb7_404a_85b7_4d3a9531fb6a.autogen.disk"
Mar 21 20:34:05 mschwore71 influxd[21484]: [httpd] 127.0.0.1 - null [21/Mar/2019:20:34:05 +0000] "POST /write?db=influx_tests_c7ccabbe_7bb7_404a_85b7_4d3a9531fb6a&rp=autogen&precision=n&consistency=one HTTP/1.1" 204 0 "-" "okhttp/3.5.0" ab8b607d-4c18-11e9-8c00-00505684bea9 13274
```
Stepping through the debugger, I see something like this happening (on my machine at least)
1. [Main Thread] Write buffer is filled to 1k points
2. [Main Thread] Schedule worker thread to write 1k points
3. [Main Thread] Another 1 point is added to the write buffer
4. [Main Thread] Disable batch
5. [Main Thread] Write remaining 1 point in buffer
6. [Main Thread] Query for points
7. [Worker Thread] Write 1k points
Also to note, this causes influxdb-java to write points out of order to influxdb.
Guida per i contributori
Nessuna guida per i contributori indicizzata per questo repository
Direzione di ricerca
Inizia riproducendo lo scenario di unit-test fornito con BatchOptions.DEFAULTS, quindi analizza il batch worker e il comportamento di disableBatch coinvolti nel flush dei punti. Verifica che la disabilitazione del batching attenda la scrittura pianificata di 1.000 punti prima dell'esecuzione della query immediata e che la query restituisca tutti i 1.001 punti senza scritture fuori ordine.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Valutazione
- Stack tecnologico
- java
- Ambito
- databases
- Tipo di issue
- Bug
- Difficoltà
- 3/5
- Tempo stimato
- 1-2 giorni
- Stato di attività
- Ferma
- Chiarezza
- Abbastanza chiara
- Idoneità per principianti
- 35/100