davidmoten / davidmoten/rxjava-jdbc

Connections not released after each requests ?

Open
#50 5 comments 0 reactions 0 assignees View on GitHub
question
Dominant language
Java
Stars
801
Forks
114
Avg merge
1m
Merged PRs (30d)
3

Description

Hi,

I'm having some problem on this observable. I'm using rxjava-jdbc 0.6.8 with a min size 1 and a max size 2 for the Hikari pool.

``` java
updateObservable = Observable.interval(2, 2, TimeUnit.SECONDS)
.onBackpressureLatest()
.flatMap(i -> db
.select("SELECT * FROM elements")
.autoMap(Element.class)
)
.compose(service.asTransformer())
.doOnNext(t -> System.out.println("updated " + t))
.map(e -> Observable. just(e.part1, e.part2, new Timestamp(e.updated_at), e.id))
.lift(db
.update("UPDATE elements SET part1 = ?, part2 = ?, updated_at = ? WHERE id = ?")
.parameterListOperator()
)
.compose(Observable::merge)
.doOnNext(t -> System.out.println("done " + t))
.retry((i, t) -> {
if (i < RETRIES) {
L.warn("Something went wrong. Retry (" + i + "/" + RETRIES + ")...", t);
return true;
}
else {
L.error("Something is still wrong after " + RETRIES + " retries. Fix it. " + NAME + " stopped.", t);
return false;
}
})
.onErrorResumeNext(t -> Observable.empty());
```

Here are the logs this shows

```
updated Element [id=169900]
done 1
updated Element [id=169901]
2015-12-11 11:38:59 WARN ElementUpdater:66 - Something went wrong. Retry (1/100)...
com.github.davidmoten.rx.jdbc.exceptions.SQLRuntimeException: java.sql.SQLTransientConnectionException: HikariPool-0 - Connection is not available, request timed out after 30000ms.
at com.github.davidmoten.rx.jdbc.ConnectionProviderPooled.get(ConnectionProviderPooled.java:92)
at com.github.davidmoten.rx.jdbc.QueryUpdateOnSubscribe.getConnection(QueryUpdateOnSubscribe.java:126)
at com.github.davidmoten.rx.jdbc.QueryUpdateOnSubscribe.call(QueryUpdateOnSubscribe.java:79)
at com.github.davidmoten.rx.jdbc.QueryUpdateOnSubscribe.call(QueryUpdateOnSubscribe.java:21)
at rx.Observable.unsafeSubscribe(Observable.java:8171)
at rx.internal.operators.OperatorSubscribeOn$1$1.call(OperatorSubscribeOn.java:62)
at rx.schedulers.TrampolineScheduler$InnerCurrentThreadScheduler.enqueue(TrampolineScheduler.java:77)
at rx.schedulers.TrampolineScheduler$InnerCurrentThreadScheduler.schedule(TrampolineScheduler.java:56)
at rx.internal.operators.OperatorSubscribeOn$1.onNext(OperatorSubscribeOn.java:57)
at rx.internal.operators.OperatorSubscribeOn$1.onNext(OperatorSubscribeOn.java:43)
at rx.internal.util.ScalarSynchronousObservable$1.call(ScalarSynchronousObservable.java:46)
at rx.internal.util.ScalarSynchronousObservable$1.call(ScalarSynchronousObservable.java:35)
at rx.Observable$2.call(Observable.java:162)
at rx.Observable$2.call(Observable.java:154)
at rx.Observable.unsafeSubscribe(Observable.java:8171)
at rx.internal.operators.OperatorConcat$ConcatSubscriber.subscribeNext(OperatorConcat.java:172)
at rx.internal.operators.OperatorConcat$ConcatSubscriber.onNext(OperatorConcat.java:136)
at rx.internal.operators.OperatorConcat$ConcatSubscriber.onNext(OperatorConcat.java:79)
at rx.internal.operators.OperatorMap$1.onNext(OperatorMap.java:54)
at rx.internal.operators.OperatorBufferWithSize$1.onNext(OperatorBufferWithSize.java:104)
at rx.observers.SerializedObserver.onNext(SerializedObserver.java:95)
at rx.observers.SerializedSubscriber.onNext(SerializedSubscriber.java:95)
at rx.internal.operators.OperatorConcat$ConcatInnerSubscriber.onNext(OperatorConcat.java:199)
at rx.internal.operators.OnSubscribeFromIterable$IterableProducer.slowpath(OnSubscribeFromIterable.java:97)
at rx.internal.operators.OnSubscribeFromIterable$IterableProducer.request(OnSubscribeFromIterable.java:73)
at rx.internal.producers.ProducerArbiter.setProducer(ProducerArbiter.java:126)
at rx.internal.operators.OperatorConcat$ConcatInnerSubscriber.setProducer(OperatorConcat.java:222)
at rx.internal.operators.OnSubscribeFromIterable.call(OnSubscribeFromIterable.java:49)
at rx.internal.operators.OnSubscribeFromIterable.call(OnSubscribeFromIterable.java:32)
at rx.Observable.unsafeSubscribe(Observable.java:8171)
at rx.internal.operators.OperatorConcat$ConcatSubscriber.subscribeNext(OperatorConcat.java:172)
at rx.internal.operators.OperatorConcat$ConcatSubscriber.onNext(OperatorConcat.java:136)
at rx.internal.operators.OperatorConcat$ConcatSubscriber.onNext(OperatorConcat.java:79)
at rx.internal.operators.OperatorMap$1.onNext(OperatorMap.java:54)
at rx.observers.SerializedObserver.onNext(SerializedObserver.java:95)
at rx.observers.SerializedSubscriber.onNext(SerializedSubscriber.java:95)
at rx.internal.operators.OperatorConcat$ConcatInnerSubscriber.onNext(OperatorConcat.java:199)
at rx.observers.SerializedObserver.onNext(SerializedObserver.java:95)
at rx.observers.SerializedSubscriber.onNext(SerializedSubscriber.java:95)
at rx.internal.operators.OperatorConcat$ConcatInnerSubscriber.onNext(OperatorConcat.java:199)
at rx.internal.operators.OperatorMap$1.onNext(OperatorMap.java:54)
at rx.internal.util.ScalarSynchronousObservable$1.call(ScalarSynchronousObservable.java:46)
at rx.internal.util.ScalarSynchronousObservable$1.call(ScalarSynchronousObservable.java:35)
at rx.Observable$2.call(Observable.java:162)
at rx.Observable$2.call(Observable.java:154)
at rx.Observable.unsafeSubscribe(Observable.java:8171)
at rx.internal.operators.OperatorConcat$ConcatSubscriber.subscribeNext(OperatorConcat.java:172)
at rx.internal.operators.OperatorConcat$ConcatSubscriber.onNext(OperatorConcat.java:136)
at rx.internal.operators.OperatorConcat$ConcatSubscriber.onNext(OperatorConcat.java:79)
at rx.internal.operators.OnSubscribeFromIterable$IterableProducer.slowpath(OnSubscribeFromIterable.java:97)
at rx.internal.operators.OnSubscribeFromIterable$IterableProducer.request(OnSubscribeFromIterable.java:73)
at rx.Subscriber.setProducer(Subscriber.java:211)
at rx.internal.operators.OnSubscribeFromIterable.call(OnSubscribeFromIterable.java:49)
at rx.internal.operators.OnSubscribeFromIterable.call(OnSubscribeFromIterable.java:32)
at rx.Observable$2.call(Observable.java:162)
at rx.Observable$2.call(Observable.java:154)
at rx.Observable.unsafeSubscribe(Observable.java:8171)
at rx.internal.operators.OperatorConcat$ConcatSubscriber.subscribeNext(OperatorConcat.java:172)
at rx.internal.operators.OperatorConcat$ConcatSubscriber.onNext(OperatorConcat.java:136)
at rx.internal.operators.OperatorConcat$ConcatSubscriber.onNext(OperatorConcat.java:79)
at rx.internal.operators.OnSubscribeFromIterable$IterableProducer.slowpath(OnSubscribeFromIterable.java:97)
at rx.internal.operators.OnSubscribeFromIterable$IterableProducer.request(OnSubscribeFromIterable.java:73)
at rx.Subscriber.setProducer(Subscriber.java:211)
at rx.internal.operators.OnSubscribeFromIterable.call(OnSubscribeFromIterable.java:49)
at rx.internal.operators.OnSubscribeFromIterable.call(OnSubscribeFromIterable.java:32)
at rx.Observable$2.call(Observable.java:162)
at rx.Observable$2.call(Observable.java:154)
at rx.Observable$2.call(Observable.java:162)
at rx.Observable$2.call(Observable.java:154)
at rx.Observable$2.call(Observable.java:162)
at rx.Observable$2.call(Observable.java:154)
at rx.Observable$2.call(Observable.java:162)
at rx.Observable$2.call(Observable.java:154)
at rx.Observable$2.call(Observable.java:162)
at rx.Observable$2.call(Observable.java:154)
at rx.Observable$2.call(Observable.java:162)
at rx.Observable$2.call(Observable.java:154)
at rx.Observable.unsafeSubscribe(Observable.java:8171)
at rx.internal.operators.OperatorMerge$MergeSubscriber.onNext(OperatorMerge.java:232)
at rx.internal.operators.OperatorMerge$MergeSubscriber.onNext(OperatorMerge.java:142)
at rx.internal.operators.OperatorMap$1.onNext(OperatorMap.java:54)
at rx.internal.operators.OperatorDoOnEach$1.onNext(OperatorDoOnEach.java:85)
at rx.internal.operators.OperatorMerge$MergeSubscriber.emitScalar(OperatorMerge.java:365)
at rx.internal.operators.OperatorMerge$MergeSubscriber.tryEmit(OperatorMerge.java:327)
at rx.internal.operators.OperatorMerge$InnerSubscriber.onNext(OperatorMerge.java:804)
at rx.internal.operators.OperatorFilter$1.onNext(OperatorFilter.java:54)
at rx.internal.operators.OperatorMap$1.onNext(OperatorMap.java:54)
at rx.internal.operators.OperatorOnErrorResumeNextViaFunction$1.onNext(OperatorOnErrorResumeNextViaFunction.java:111)
at rx.internal.operators.OperatorMap$1.onNext(OperatorMap.java:54)
at rx.internal.operators.OperatorMerge$MergeSubscriber.emitScalar(OperatorMerge.java:365)
at rx.internal.operators.OperatorMerge$MergeSubscriber.tryEmit(OperatorMerge.java:327)
at rx.internal.operators.OperatorMerge$InnerSubscriber.onNext(OperatorMerge.java:804)
at io.reactivex.netty.protocol.http.UnicastContentSubject$AutoReleaseByteBufOperator$1.onNext(UnicastContentSubject.java:262)
at rx.internal.operators.NotificationLite.accept(NotificationLite.java:150)
at rx.internal.operators.BufferUntilSubscriber.emit(BufferUntilSubscriber.java:145)
at rx.internal.operators.BufferUntilSubscriber.onNext(BufferUntilSubscriber.java:178)
at io.reactivex.netty.protocol.http.UnicastContentSubject.onNext(UnicastContentSubject.java:286)
at io.reactivex.netty.protocol.http.client.ClientRequestResponseConverter.invokeContentOnNext(ClientRequestResponseConverter.java:248)
at io.reactivex.netty.protocol.http.client.ClientRequestResponseConverter.channelRead(ClientRequestResponseConverter.java:141)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:242)
at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:147)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:283)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
at io.reactivex.netty.metrics.BytesInspector.channelRead(BytesInspector.java:59)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:847)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.sql.SQLTransientConnectionException: HikariPool-0 - Connection is not available, request timed out after 30000ms.
at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:196)
at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:148)
at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:99)
at com.github.davidmoten.rx.jdbc.ConnectionProviderPooled.get(ConnectionProviderPooled.java:90)
... 123 more
updated Element [id=169901]
done 1
updated Element [id=169902]
done 1
updated Element [id=169903]
2015-12-11 11:39:35 WARN ElementUpdater:66 - Something went wrong. Retry (2/100)...
com.github.davidmoten.rx.jdbc.exceptions.SQLRuntimeException: java.sql.SQLTransientConnectionException: HikariPool-0 - Connection is not available, request timed out after 30000ms.
[...] Same stack trace
updated Element [id=169903]
done 1
...
```

So each time it does 2 requests, it freezes and times out at 30 seconds (I guess cause all the connections are used). It then retries , succeeds at the id it failed on, succeeds at the next and freezes once again when already 2 requests have been done. And so on.

I must not be understanding something about the pools here. Please let me know if i'm doing something wrong.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.