ISISComputingGroup / ISISComputingGroup/IBEX
Block Archive: Occasionally restarting
- Dominant language
- No language data
- Stars
- 6
- Forks
- 2
- Avg merge
- 16h 40m
- Merged PRs (30d)
- 2
Description
As an instrument scientist I would like to reliably get the values of my blocks. Currently the block archive engine is dropping out about every 5 minutes with the following error:
```
[2021-04-30 16:49:38] 2021-04-30 16:49:38.342 INFO [Thread 1] org.csstudio.archive.engine.Application (start) - ArchiveEngine ending
[2021-04-30 16:49:38] 2021-04-30 16:49:38.343 INFO [Thread 1] org.csstudio.archive.engine.model.EngineModel (stop) - Stopping scanner
[2021-04-30 16:49:38] 2021-04-30 16:49:38.489 INFO [Thread 207] org.csstudio.archive.engine.scanner.ScanThread (run) - Scan Thread ends
[2021-04-30 16:49:38] 2021-04-30 16:49:38.489 INFO [Thread 1] org.csstudio.archive.engine.model.EngineModel (stop) - Stopping archive groups
[2021-04-30 16:49:38] 2021-04-30 16:49:38.501 INFO [Thread 1] org.csstudio.archive.engine.model.EngineModel (stop) - Stopping writer
[2021-04-30 16:49:38] 2021-04-30 16:49:38.502 INFO [Thread 206] org.csstudio.archive.engine.model.WriteThread (run) - WriteThread exists
[2021-04-30 16:49:38] 2021-04-30 16:49:38.561 INFO [Thread 1] org.csstudio.archive.engine.Application (start) - Reading configuration 'block_engine'
```
This will lead to intermittent data in the database:

Additionally, if this happens at the same time that JSON_bourne is attempting to get data it will then cause the following error:
```
[2021-04-30 16:48:27] 2021-04-30 16:48:27.737 WARNING [Thread 251] org.csstudio.archive.engine.server.Servlet (doGet) - HTTP Server exception
[2021-04-30 16:48:27] java.lang.ArrayIndexOutOfBoundsException: Index 81 out of bounds for length 0
[2021-04-30 16:48:27] at java.base/java.util.concurrent.CopyOnWriteArrayList.elementAt(CopyOnWriteArrayList.java:385)
[2021-04-30 16:48:27] at java.base/java.util.concurrent.CopyOnWriteArrayList.get(CopyOnWriteArrayList.java:398)
[2021-04-30 16:48:27] at org.csstudio.archive.engine.model.ArchiveGroup.getChannel(ArchiveGroup.java:97)
[2021-04-30 16:48:27] at org.csstudio.archive.engine.server.json.JSONGroupResponse.fillResponse(JSONGroupResponse.java:52)
[2021-04-30 16:48:27] at org.csstudio.archive.engine.server.Servlet.doGet(Servlet.java:60)
[2021-04-30 16:48:27] at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
[2021-04-30 16:48:27] at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
[2021-04-30 16:48:27] at org.eclipse.equinox.http.servlet.internal.HttpServiceRuntimeImpl$LegacyServlet.service(HttpServiceRuntimeImpl.java:1240)
[2021-04-30 16:48:27] at org.eclipse.equinox.http.servlet.internal.registration.EndpointRegistration.service(EndpointRegistration.java:151)
[2021-04-30 16:48:27] at org.eclipse.equinox.http.servlet.internal.servlet.ResponseStateHandler.processRequest(ResponseStateHandler.java:65)
[2021-04-30 16:48:27] at org.eclipse.equinox.http.servlet.internal.context.DispatchTargets.doDispatch(DispatchTargets.java:134)
[2021-04-30 16:48:27] at org.eclipse.equinox.http.servlet.internal.servlet.ProxyServlet.service(ProxyServlet.java:103)
[2021-04-30 16:48:27] at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
[2021-04-30 16:48:27] at org.eclipse.equinox.http.jetty.internal.HttpServerManager$InternalHttpServiceServlet.service(HttpServerManager.java:300)
[2021-04-30 16:48:27] at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:867)
[2021-04-30 16:48:27] at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:542)
[2021-04-30 16:48:27] at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
[2021-04-30 16:48:27] at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1588)
[2021-04-30 16:48:27] at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
[2021-04-30 16:48:27] at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345)
[2021-04-30 16:48:27] at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
[2021-04-30 16:48:27] at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
[2021-04-30 16:48:27] at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1557)
[2021-04-30 16:48:27] at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
[2021-04-30 16:48:27] at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247)
[2021-04-30 16:48:27] at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
[2021-04-30 16:48:27] at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
[2021-04-30 16:48:27] at org.eclipse.jetty.server.Server.handle(Server.java:502)
[2021-04-30 16:48:27] at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364)
[2021-04-30 16:48:27] at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
[2021-04-30 16:48:27] at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
[2021-04-30 16:48:27] at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
[2021-04-30 16:48:27] at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
[2021-04-30 16:48:27] at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
[2021-04-30 16:48:27] at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
[2021-04-30 16:48:27] at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
[2021-04-30 16:48:27] at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
[2021-04-30 16:48:27] at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
[2021-04-30 16:48:27] at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
[2021-04-30 16:48:27] at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
[2021-04-30 16:48:27] at java.base/java.lang.Thread.run(Thread.java:834)
```
Which can cause the webdashboard to crash as seen in #6464 .
## Acceptance Criteria
* The block archive engine no longer occasionally fails
* There is a system test that would have covered this
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.