davidmoten / davidmoten/rxjava-jdbc
Connections not released after each requests ?
- Dominant language
- Java
- Stars
- 801
- Forks
- 114
- Avg merge
- 1m
- Merged PRs (30d)
- 3
Description
Hi,
I'm having some problem on this observable. I'm using rxjava-jdbc 0.6.8 with a min size 1 and a max size 2 for the Hikari pool.
``` java
updateObservable = Observable.interval(2, 2, TimeUnit.SECONDS)
.onBackpressureLatest()
.flatMap(i -> db
.select("SELECT * FROM elements")
.autoMap(Element.class)
)
.compose(service.asTransformer())
.doOnNext(t -> System.out.println("updated " + t))
.map(e -> Observable. just(e.part1, e.part2, new Timestamp(e.updated_at), e.id))
.lift(db
.update("UPDATE elements SET part1 = ?, part2 = ?, updated_at = ? WHERE id = ?")
.parameterListOperator()
)
.compose(Observable::merge)
.doOnNext(t -> System.out.println("done " + t))
.retry((i, t) -> {
if (i < RETRIES) {
L.warn("Something went wrong. Retry (" + i + "/" + RETRIES + ")...", t);
return true;
}
else {
L.error("Something is still wrong after " + RETRIES + " retries. Fix it. " + NAME + " stopped.", t);
return false;
}
})
.onErrorResumeNext(t -> Observable.empty());
```
Here are the logs this shows
```
updated Element [id=169900]
done 1
updated Element [id=169901]
2015-12-11 11:38:59 WARN ElementUpdater:66 - Something went wrong. Retry (1/100)...
com.github.davidmoten.rx.jdbc.exceptions.SQLRuntimeException: java.sql.SQLTransientConnectionException: HikariPool-0 - Connection is not available, request timed out after 30000ms.
at com.github.davidmoten.rx.jdbc.ConnectionProviderPooled.get(ConnectionProviderPooled.java:92)
at com.github.davidmoten.rx.jdbc.QueryUpdateOnSubscribe.getConnection(QueryUpdateOnSubscribe.java:126)
at com.github.davidmoten.rx.jdbc.QueryUpdateOnSubscribe.call(QueryUpdateOnSubscribe.java:79)
at com.github.davidmoten.rx.jdbc.QueryUpdateOnSubscribe.call(QueryUpdateOnSubscribe.java:21)
at rx.Observable.unsafeSubscribe(Observable.java:8171)
at rx.internal.operators.OperatorSubscribeOn$1$1.call(OperatorSubscribeOn.java:62)
at rx.schedulers.TrampolineScheduler$InnerCurrentThreadScheduler.enqueue(TrampolineScheduler.java:77)
at rx.schedulers.TrampolineScheduler$InnerCurrentThreadScheduler.schedule(TrampolineScheduler.java:56)
at rx.internal.operators.OperatorSubscribeOn$1.onNext(OperatorSubscribeOn.java:57)
at rx.internal.operators.OperatorSubscribeOn$1.onNext(OperatorSubscribeOn.java:43)
at rx.internal.util.ScalarSynchronousObservable$1.call(ScalarSynchronousObservable.java:46)
at rx.internal.util.ScalarSynchronousObservable$1.call(ScalarSynchronousObservable.java:35)
at rx.Observable$2.call(Observable.java:162)
at rx.Observable$2.call(Observable.java:154)
at rx.Observable.unsafeSubscribe(Observable.java:8171)
at rx.internal.operators.OperatorConcat$ConcatSubscriber.subscribeNext(OperatorConcat.java:172)
at rx.internal.operators.OperatorConcat$ConcatSubscriber.onNext(OperatorConcat.java:136)
at rx.internal.operators.OperatorConcat$ConcatSubscriber.onNext(OperatorConcat.java:79)
at rx.internal.operators.OperatorMap$1.onNext(OperatorMap.java:54)
at rx.internal.operators.OperatorBufferWithSize$1.onNext(OperatorBufferWithSize.java:104)
at rx.observers.SerializedObserver.onNext(SerializedObserver.java:95)
at rx.observers.SerializedSubscriber.onNext(SerializedSubscriber.java:95)
at rx.internal.operators.OperatorConcat$ConcatInnerSubscriber.onNext(OperatorConcat.java:199)
at rx.internal.operators.OnSubscribeFromIterable$IterableProducer.slowpath(OnSubscribeFromIterable.java:97)
at rx.internal.operators.OnSubscribeFromIterable$IterableProducer.request(OnSubscribeFromIterable.java:73)
at rx.internal.producers.ProducerArbiter.setProducer(ProducerArbiter.java:126)
at rx.internal.operators.OperatorConcat$ConcatInnerSubscriber.setProducer(OperatorConcat.java:222)
at rx.internal.operators.OnSubscribeFromIterable.call(OnSubscribeFromIterable.java:49)
at rx.internal.operators.OnSubscribeFromIterable.call(OnSubscribeFromIterable.java:32)
at rx.Observable.unsafeSubscribe(Observable.java:8171)
at rx.internal.operators.OperatorConcat$ConcatSubscriber.subscribeNext(OperatorConcat.java:172)
at rx.internal.operators.OperatorConcat$ConcatSubscriber.onNext(OperatorConcat.java:136)
at rx.internal.operators.OperatorConcat$ConcatSubscriber.onNext(OperatorConcat.java:79)
at rx.internal.operators.OperatorMap$1.onNext(OperatorMap.java:54)
at rx.observers.SerializedObserver.onNext(SerializedObserver.java:95)
at rx.observers.SerializedSubscriber.onNext(SerializedSubscriber.java:95)
at rx.internal.operators.OperatorConcat$ConcatInnerSubscriber.onNext(OperatorConcat.java:199)
at rx.observers.SerializedObserver.onNext(SerializedObserver.java:95)
at rx.observers.SerializedSubscriber.onNext(SerializedSubscriber.java:95)
at rx.internal.operators.OperatorConcat$ConcatInnerSubscriber.onNext(OperatorConcat.java:199)
at rx.internal.operators.OperatorMap$1.onNext(OperatorMap.java:54)
at rx.internal.util.ScalarSynchronousObservable$1.call(ScalarSynchronousObservable.java:46)
at rx.internal.util.ScalarSynchronousObservable$1.call(ScalarSynchronousObservable.java:35)
at rx.Observable$2.call(Observable.java:162)
at rx.Observable$2.call(Observable.java:154)
at rx.Observable.unsafeSubscribe(Observable.java:8171)
at rx.internal.operators.OperatorConcat$ConcatSubscriber.subscribeNext(OperatorConcat.java:172)
at rx.internal.operators.OperatorConcat$ConcatSubscriber.onNext(OperatorConcat.java:136)
at rx.internal.operators.OperatorConcat$ConcatSubscriber.onNext(OperatorConcat.java:79)
at rx.internal.operators.OnSubscribeFromIterable$IterableProducer.slowpath(OnSubscribeFromIterable.java:97)
at rx.internal.operators.OnSubscribeFromIterable$IterableProducer.request(OnSubscribeFromIterable.java:73)
at rx.Subscriber.setProducer(Subscriber.java:211)
at rx.internal.operators.OnSubscribeFromIterable.call(OnSubscribeFromIterable.java:49)
at rx.internal.operators.OnSubscribeFromIterable.call(OnSubscribeFromIterable.java:32)
at rx.Observable$2.call(Observable.java:162)
at rx.Observable$2.call(Observable.java:154)
at rx.Observable.unsafeSubscribe(Observable.java:8171)
at rx.internal.operators.OperatorConcat$ConcatSubscriber.subscribeNext(OperatorConcat.java:172)
at rx.internal.operators.OperatorConcat$ConcatSubscriber.onNext(OperatorConcat.java:136)
at rx.internal.operators.OperatorConcat$ConcatSubscriber.onNext(OperatorConcat.java:79)
at rx.internal.operators.OnSubscribeFromIterable$IterableProducer.slowpath(OnSubscribeFromIterable.java:97)
at rx.internal.operators.OnSubscribeFromIterable$IterableProducer.request(OnSubscribeFromIterable.java:73)
at rx.Subscriber.setProducer(Subscriber.java:211)
at rx.internal.operators.OnSubscribeFromIterable.call(OnSubscribeFromIterable.java:49)
at rx.internal.operators.OnSubscribeFromIterable.call(OnSubscribeFromIterable.java:32)
at rx.Observable$2.call(Observable.java:162)
at rx.Observable$2.call(Observable.java:154)
at rx.Observable$2.call(Observable.java:162)
at rx.Observable$2.call(Observable.java:154)
at rx.Observable$2.call(Observable.java:162)
at rx.Observable$2.call(Observable.java:154)
at rx.Observable$2.call(Observable.java:162)
at rx.Observable$2.call(Observable.java:154)
at rx.Observable$2.call(Observable.java:162)
at rx.Observable$2.call(Observable.java:154)
at rx.Observable$2.call(Observable.java:162)
at rx.Observable$2.call(Observable.java:154)
at rx.Observable.unsafeSubscribe(Observable.java:8171)
at rx.internal.operators.OperatorMerge$MergeSubscriber.onNext(OperatorMerge.java:232)
at rx.internal.operators.OperatorMerge$MergeSubscriber.onNext(OperatorMerge.java:142)
at rx.internal.operators.OperatorMap$1.onNext(OperatorMap.java:54)
at rx.internal.operators.OperatorDoOnEach$1.onNext(OperatorDoOnEach.java:85)
at rx.internal.operators.OperatorMerge$MergeSubscriber.emitScalar(OperatorMerge.java:365)
at rx.internal.operators.OperatorMerge$MergeSubscriber.tryEmit(OperatorMerge.java:327)
at rx.internal.operators.OperatorMerge$InnerSubscriber.onNext(OperatorMerge.java:804)
at rx.internal.operators.OperatorFilter$1.onNext(OperatorFilter.java:54)
at rx.internal.operators.OperatorMap$1.onNext(OperatorMap.java:54)
at rx.internal.operators.OperatorOnErrorResumeNextViaFunction$1.onNext(OperatorOnErrorResumeNextViaFunction.java:111)
at rx.internal.operators.OperatorMap$1.onNext(OperatorMap.java:54)
at rx.internal.operators.OperatorMerge$MergeSubscriber.emitScalar(OperatorMerge.java:365)
at rx.internal.operators.OperatorMerge$MergeSubscriber.tryEmit(OperatorMerge.java:327)
at rx.internal.operators.OperatorMerge$InnerSubscriber.onNext(OperatorMerge.java:804)
at io.reactivex.netty.protocol.http.UnicastContentSubject$AutoReleaseByteBufOperator$1.onNext(UnicastContentSubject.java:262)
at rx.internal.operators.NotificationLite.accept(NotificationLite.java:150)
at rx.internal.operators.BufferUntilSubscriber.emit(BufferUntilSubscriber.java:145)
at rx.internal.operators.BufferUntilSubscriber.onNext(BufferUntilSubscriber.java:178)
at io.reactivex.netty.protocol.http.UnicastContentSubject.onNext(UnicastContentSubject.java:286)
at io.reactivex.netty.protocol.http.client.ClientRequestResponseConverter.invokeContentOnNext(ClientRequestResponseConverter.java:248)
at io.reactivex.netty.protocol.http.client.ClientRequestResponseConverter.channelRead(ClientRequestResponseConverter.java:141)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:242)
at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:147)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:283)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
at io.reactivex.netty.metrics.BytesInspector.channelRead(BytesInspector.java:59)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:847)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.sql.SQLTransientConnectionException: HikariPool-0 - Connection is not available, request timed out after 30000ms.
at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:196)
at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:148)
at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:99)
at com.github.davidmoten.rx.jdbc.ConnectionProviderPooled.get(ConnectionProviderPooled.java:90)
... 123 more
updated Element [id=169901]
done 1
updated Element [id=169902]
done 1
updated Element [id=169903]
2015-12-11 11:39:35 WARN ElementUpdater:66 - Something went wrong. Retry (2/100)...
com.github.davidmoten.rx.jdbc.exceptions.SQLRuntimeException: java.sql.SQLTransientConnectionException: HikariPool-0 - Connection is not available, request timed out after 30000ms.
[...] Same stack trace
updated Element [id=169903]
done 1
...
```
So each time it does 2 requests, it freezes and times out at 30 seconds (I guess cause all the connections are used). It then retries , succeeds at the id it failed on, succeeds at the next and freezes once again when already 2 requests have been done. And so on.
I must not be understanding something about the pools here. Please let me know if i'm doing something wrong.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.