HTML NPE in sampler if dimensionsSpecs is not set
- Dominant language
- Java
- Stars
- 14.1k
- Forks
- 3.8k
- Avg merge
- 2d 58m
- Merged PRs (30d)
- 233
Description
### Affected Version
Latest master
### Description
NPE in sampler if missing dimensionsSpec
Submit:
```json
{
"type": "index",
"spec": {
"type": "index",
"ioConfig": {
"type": "index",
"inputSource": {
"type": "inline",
"data": "{\"time\":\"2015-09-12T00:46:58.771Z\",\"channel\":\"#en.wikipedia\",\"cityName\":null,\"comment\":\"added project\",\"countryIsoCode\":null,\"countryName\":null,\"isAnonymous\":false,\"isMinor\":false,\"isNew\":false,\"isRobot\":false,\"isUnpatrolled\":false,\"metroCode\":null,\"namespace\":\"Talk\",\"page\":\"Talk:Oswald Tilghman\",\"regionIsoCode\":null,\"regionName\":null,\"user\":\"GELongstreet\",\"delta\":36,\"added\":36,\"deleted\":0}\n{\"time\":\"2015-09-12T00:47:00.496Z\",\"channel\":\"#ca.wikipedia\",\"cityName\":null,\"comment\":\"Robot inserta {{Commonscat}} que enllaça amb [[commons:category:Rallicula]]\",\"countryIsoCode\":null,\"countryName\":null,\"isAnonymous\":false,\"isMinor\":true,\"isNew\":false,\"isRobot\":true,\"isUnpatrolled\":false,\"metroCode\":null,\"namespace\":\"Main\",\"page\":\"Rallicula\",\"regionIsoCode\":null,\"regionName\":null,\"user\":\"PereBot\",\"delta\":17,\"added\":17,\"deleted\":0}\n{\"time\":\"2015-09-12T00:47:05.474Z\",\"channel\":\"#en.wikipedia\",\"cityName\":\"Auburn\",\"comment\":\"/* Status of peremptory norms under international law */ fixed spelling of 'Wimbledon'\",\"countryIsoCode\":\"AU\",\"countryName\":\"Australia\",\"isAnonymous\":true,\"isMinor\":false,\"isNew\":false,\"isRobot\":false,\"isUnpatrolled\":false,\"metroCode\":null,\"namespace\":\"Main\",\"page\":\"Peremptory norm\",\"regionIsoCode\":\"NSW\",\"regionName\":\"New South Wales\",\"user\":\"60.225.66.142\",\"delta\":0,\"added\":0,\"deleted\":0}\n{\"time\":\"2015-09-12T00:47:08.770Z\",\"channel\":\"#vi.wikipedia\",\"cityName\":null,\"comment\":\"fix Lỗi CS1: ngày tháng\",\"countryIsoCode\":null,\"countryName\":null,\"isAnonymous\":false,\"isMinor\":true,\"isNew\":false,\"isRobot\":true,\"isUnpatrolled\":false,\"metroCode\":null,\"namespace\":\"Main\",\"page\":\"Apamea abruzzorum\",\"regionIsoCode\":null,\"regionName\":null,\"user\":\"Cheers!-bot\",\"delta\":18,\"added\":18,\"deleted\":0}\n{\"time\":\"2015-09-12T00:47:11.862Z\",\"channel\":\"#vi.wikipedia\",\"cityName\":null,\"comment\":\"clean up using [[Project:AWB|AWB]]\",\"countryIsoCode\":null,\"countryName\":null,\"isAnonymous\":false,\"isMinor\":false,\"isNew\":false,\"isRobot\":true,\"isUnpatrolled\":false,\"metroCode\":null,\"namespace\":\"Main\",\"page\":\"Atractus flammigerus\",\"regionIsoCode\":null,\"regionName\":null,\"user\":\"ThitxongkhoiAWB\",\"delta\":18,\"added\":18,\"deleted\":0}\n{\"time\":\"2015-09-12T00:47:13.987Z\",\"channel\":\"#vi.wikipedia\",\"cityName\":null,\"comment\":\"clean up using [[Project:AWB|AWB]]\",\"countryIsoCode\":null,\"countryName\":null,\"isAnonymous\":false,\"isMinor\":false,\"isNew\":false,\"isRobot\":true,\"isUnpatrolled\":false,\"metroCode\":null,\"namespace\":\"Main\",\"page\":\"Agama mossambica\",\"regionIsoCode\":null,\"regionName\":null,\"user\":\"ThitxongkhoiAWB\",\"delta\":18,\"added\":18,\"deleted\":0}\n{\"time\":\"2015-09-12T00:47:17.009Z\",\"channel\":\"#ca.wikipedia\",\"cityName\":null,\"comment\":\"/* Imperi Austrohongarès */\",\"countryIsoCode\":null,\"countryName\":null,\"isAnonymous\":false,\"isMinor\":false,\"isNew\":false,\"isRobot\":false,\"isUnpatrolled\":false,\"metroCode\":null,\"namespace\":\"Main\",\"page\":\"Campanya dels Balcans (1914-1918)\",\"regionIsoCode\":null,\"regionName\":null,\"user\":\"Jaumellecha\",\"delta\":-20,\"added\":0,\"deleted\":20}\n{\"time\":\"2015-09-12T00:47:19.591Z\",\"channel\":\"#en.wikipedia\",\"cityName\":null,\"comment\":\"adding comment on notability and possible COI\",\"countryIsoCode\":null,\"countryName\":null,\"isAnonymous\":false,\"isMinor\":false,\"isNew\":true,\"isRobot\":false,\"isUnpatrolled\":true,\"metroCode\":null,\"namespace\":\"Talk\",\"page\":\"Talk:Dani Ploeger\",\"regionIsoCode\":null,\"regionName\":null,\"user\":\"New Media Theorist\",\"delta\":345,\"added\":345,\"deleted\":0}\n{\"time\":\"2015-09-12T00:47:21.578Z\",\"channel\":\"#en.wikipedia\",\"cityName\":null,\"comment\":\"Copying assessment table to wiki\",\"countryIsoCode\":null,\"countryName\":null,\"isAnonymous\":false,\"isMinor\":false,\"isNew\":false,\"isRobot\":true,\"isUnpatrolled\":false,\"metroCode\":null,\"namespace\":\"User\",\"page\":\"User:WP 1.0 bot/Tables/Project/Pubs\",\"regionIsoCode\":null,\"regionName\":null,\"user\":\"WP 1.0 bot\",\"delta\":121,\"added\":121,\"deleted\":0}\n{\"time\":\"2015-09-12T00:47:25.821Z\",\"channel\":\"#vi.wikipedia\",\"cityName\":null,\"comment\":\"clean up using [[Project:AWB|AWB]]\",\"countryIsoCode\":null,\"countryName\":null,\"isAnonymous\":false,\"isMinor\":false,\"isNew\":false,\"isRobot\":true,\"isUnpatrolled\":false,\"metroCode\":null,\"namespace\":\"Main\",\"page\":\"Agama persimilis\",\"regionIsoCode\":null,\"regionName\":null,\"user\":\"ThitxongkhoiAWB\",\"delta\":18,\"added\":18,\"deleted\":0}\n{\"time\":\"2015-09-12T00:47:29.913Z\",\"channel\":\"#en.wikipedia\",\"cityName\":null,\"comment\":\"Blank stale warning(s) and replace with {{[[template:OW|OW]]}} using [[Project:AWB|AWB]]\",\"countryIsoCode\":null,\"countryName\":null,\"isAnonymous\":false,\"isMinor\":true,\"isNew\":false,\"isRobot\":false,\"isUnpatrolled\":false,\"metroCode\":null,\"namespace\":\"User talk\",\"page\":\"User talk:161.184.95.17\",\"regionIsoCode\":null,\"regionName\":null,\"user\":\"DavidLeighEllis\",\"delta\":0,\"added\":0,\"deleted\":0}\n{\"time\":\"2015-09-12T00:47:33.004Z\",\"channel\":\"#vi.wikipedia\",\"cityName\":null,\"comment\":\"clean up using [[Project:AWB|AWB]]\",\"countryIsoCode\":null,\"countryName\":null,\"isAnonymous\":false,\"isMinor\":false,\"isNew\":false,\"isRobot\":true,\"isUnpatrolled\":false,\"metroCode\":null,\"namespace\":\"Main\",\"page\":\"Atractus edioi\",\"regionIsoCode\":null,\"regionName\":null,\"user\":\"ThitxongkhoiAWB\",\"delta\":18,\"added\":18,\"deleted\":0}\n{\"time\":\"2015-09-12T00:47:35.776Z\",\"channel\":\"#vi.wikipedia\",\"cityName\":null,\"comment\":\"Lỗi CS1: ngày tháng\",\"countryIsoCode\":null,\"countryName\":null,\"isAnonymous\":false,\"isMinor\":true,\"isNew\":false,\"isRobot\":true,\"isUnpatrolled\":false,\"metroCode\":null,\"namespace\":\"Main\",\"page\":\"Agave gentryi\",\"regionIsoCode\":null,\"regionName\":null,\"user\":\"TuHan-Bot\",\"delta\":36,\"added\":36,\"deleted\":0}\n{\"time\":\"2015-09-12T00:47:37.881Z\",\"channel\":\"#vi.wikipedia\",\"cityName\":null,\"comment\":\"clean up using [[Project:AWB|AWB]]\",\"countryIsoCode\":null,\"countryName\":null,\"isAnonymous\":false,\"isMinor\":false,\"isNew\":false,\"isRobot\":true,\"isUnpatrolled\":false,\"metroCode\":null,\"namespace\":\"Main\",\"page\":\"Agama sankaranica\",\"regionIsoCode\":null,\"regionName\":null,\"user\":\"ThitxongkhoiAWB\",\"delta\":18,\"added\":18,\"deleted\":0}\n{\"time\":\"2015-09-12T00:47:42.090Z\",\"channel\":\"#vi.wikipedia\",\"cityName\":null,\"comment\":\"fix Lỗi CS1: ngày tháng\",\"countryIsoCode\":null,\"countryName\":null,\"isAnonymous\":false,\"isMinor\":true,\"isNew\":false,\"isRobot\":true,\"isUnpatrolled\":false,\"metroCode\":null,\"namespace\":\"Main\",\"page\":\"Apamea albertae\",\"regionIsoCode\":null,\"regionName\":null,\"user\":\"Cheers!-bot\",\"delta\":18,\"added\":18,\"deleted\":0}\n{\"time\":\"2015-09-12T00:47:44.963Z\",\"channel\":\"#ru.wikipedia\",\"cityName\":null,\"comment\":\"/* Донецкая Народная Республика */\",\"countryIsoCode\":null,\"countryName\":null,\"isAnonymous\":false,\"isMinor\":false,\"isNew\":false,\"isRobot\":false,\"isUnpatrolled\":false,\"metroCode\":null,\"namespace\":\"Main\",\"page\":\"Караман, Александр Акимович\",\"regionIsoCode\":null,\"regionName\":null,\"user\":\"Камарад Че\",\"delta\":0,\"added\":0,\"deleted\":0}\n{\"time\":\"2015-09-12T00:47:47.870Z\",\"channel\":\"#vi.wikipedia\",\"cityName\":null,\"comment\":\"clean up using [[Project:AWB|AWB]]\",\"countryIsoCode\":null,\"countryName\":null,\"isAnonymous\":false,\"isMinor\":false,\"isNew\":false,\"isRobot\":true,\"isUnpatrolled\":false,\"metroCode\":null,\"namespace\":\"Main\",\"page\":\"Atractus duboisi\",\"regionIsoCode\":null,\"regionName\":null,\"user\":\"ThitxongkhoiAWB\",\"delta\":18,\"added\":18,\"deleted\":0}\n{\"time\":\"2015-09-12T00:47:50.819Z\",\"channel\":\"#en.wikipedia\",\"cityName\":null,\"comment\":\"/* Films */\",\"countryIsoCode\":null,\"countryName\":null,\"isAnonymous\":false,\"isMinor\":false,\"isNew\":false,\"isRobot\":false,\"isUnpatrolled\":false,\"metroCode\":null,\"namespace\":\"Main\",\"page\":\"Keiynan Lonsdale\",\"regionIsoCode\":null,\"regionName\":null,\"user\":\"Lg16spears\",\"delta\":-11,\"added\":0,\"deleted\":11}\n{\"time\":\"2015-09-12T00:47:53.259Z\",\"channel\":\"#ja.wikipedia\",\"cityName\":null,\"comment\":\"/* 対戦通算成績と得失点 */\",\"countryIsoCode\":null,\"countryName\":null,\"isAnonymous\":false,\"isMinor\":true,\"isNew\":false,\"isRobot\":false,\"isUnpatrolled\":false,\"metroCode\":null,\"namespace\":\"Main\",\"page\":\"アルビレックス新潟の年度別成績一覧\",\"regionIsoCode\":null,\"regionName\":null,\"user\":\"BlueMoon2662\",\"delta\":14,\"added\":14,\"deleted\":0}\n{\"time\":\"2015-09-12T00:47:56.126Z\",\"channel\":\"#en.wikipedia\",\"cityName\":null,\"comment\":\"Bot updating unblock request table ([[en:WP:PEACHY|Peachy 2.0 (alpha 8)]])\",\"countryIsoCode\":null,\"countryName\":null,\"isAnonymous\":false,\"isMinor\":false,\"isNew\":false,\"isRobot\":true,\"isUnpatrolled\":false,\"metroCode\":null,\"namespace\":\"User\",\"page\":\"User:Cyberbot I/Requests for unblock report\",\"regionIsoCode\":null,\"regionName\":null,\"user\":\"Cyberbot I\",\"delta\":-74,\"added\":0,\"deleted\":74}"
},
"inputFormat": {
"type": "json"
}
},
"dataSchema": {
"dataSource": "sample",
"timestampSpec": {
"column": "time",
"format": "iso"
},
"transformSpec": {},
"granularitySpec": {
"queryGranularity": "HOUR"
},
"metricsSpec": []
},
"tuningConfig": {
"type": "index"
}
},
"samplerConfig": {
"numRows": 500,
"timeoutMs": 15000
}
}
```
Returns:
```
Error 500 Server Error
HTTP ERROR 500
Problem accessing /druid/indexer/v1/sampler. Reason:
Server Error
Caused by:
java.lang.NullPointerException: inputRowParser
at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:229)
at org.apache.druid.segment.indexing.DataSchema.getDimensionsSpec(DataSchema.java:227)
at org.apache.druid.indexing.overlord.sampler.InputSourceSampler.buildReader(InputSourceSampler.java:200)
at org.apache.druid.indexing.overlord.sampler.InputSourceSampler.sample(InputSourceSampler.java:107)
at org.apache.druid.indexing.overlord.sampler.IndexTaskSamplerSpec.sample(IndexTaskSamplerSpec.java:94)
at org.apache.druid.indexing.overlord.sampler.SamplerResource.post(SamplerResource.java:41)
at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542)
at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473)
at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)
at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409)
at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)
at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558)
at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at com.google.inject.servlet.ServletDefinition.doServiceImpl(ServletDefinition.java:286)
at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:276)
at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:181)
at com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)
at com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:120)
at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:135)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
at org.apache.druid.server.http.RedirectFilter.doFilter(RedirectFilter.java:73)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
at org.apache.druid.server.security.PreResponseAuthorizationCheckFilter.doFilter(PreResponseAuthorizationCheckFilter.java:82)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
at org.apache.druid.server.security.AllowOptionsResourceFilter.doFilter(AllowOptionsResourceFilter.java:75)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
at org.apache.druid.server.security.AllowAllAuthenticator$1.doFilter(AllowAllAuthenticator.java:84)
at org.apache.druid.server.security.AuthenticationWrappingFilter.doFilter(AuthenticationWrappingFilter.java:59)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
at org.apache.druid.server.security.SecuritySanityCheckFilter.doFilter(SecuritySanityCheckFilter.java:86)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1340)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1242)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:740)
at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:61)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.Server.handle(Server.java:503)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
at java.lang.Thread.run(Thread.java:748)
Powered by Jetty:// 9.4.12.v20180830
```
Filing this because HTML NPEs are always considered bugs in my book. Should be a JSON exception
Contributor guide
Research direction
Start with DataSchema.getDimensionsSpec and InputSourceSampler.buildReader, then reproduce the issue using the sampler request in the report. Trace how the missing dimensionsSpec reaches the sampler and verify that the request no longer produces a NullPointerException or HTTP 500 response.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- api, backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100