BatchProcessor, going single to multithread ?

Using influxdb-java on a 4core/8thread machine to connect to a remote InfluxDB (located in the same DC), I experience very high CPU usage for the thread responsible for the BatchProcessor and suspect we could benefit from having BatchProcess use several threads.

It looks like all batched points go to a unique queue in the BatchProcessor instance and that a singleThreadScheduledExecutor() is responsible for dequeuing them.

My setup is not CPU bound (as shown by htop), nor memory nor network bound. Do you think allowing parallel queuing/unqueuing would increase capacity of influxdb-java to flush its data ?

I'm thinking about having a configurable number of queues (let's call it N), one singleThreadScheduledExecutor() each. Because the data is shared nothing we could make the batchProcess queue the data in a round-robin way, and then have their respective scheduleExecutor do the job.

Does this make sense ? Do you think is is worth ?

Inizia leggendo l’implementazione di BatchProcessor e il percorso singleThreadScheduledExecutor descritto nell’issue. Determina se code ed executor paralleli configurabili sono realizzabili senza modificare il comportamento del batching, quindi definisci misurazioni o test che mostrino se la modifica aumenta la capacità di flush.

Scritto dal modello di indicizzazione a partire dal testo della issue.