metabase / metabase/metabase

sync schedules with */minute format break the server

Open
#15,057 2 comments 0 reactions 0 assignees View on GitHub
.Team/SemanticLayer Administration/Metadata & Sync Priority:P3 Semantic Layer/Sync Type:New Feature
Dominant language
Clojure
Stars
49.3k
Forks
6.8k
Avg merge
1d 13h
Merged PRs (30d)
653

Description

**Describe the bug**
I specified a cron schedule of `0 */5 * * * ? *` so I could test running sync every 5 minutes. This is a valid cron schedule, but the server can't handle it.

**Logs**

```text
2021-03-03 22:05:26,594 ERROR middleware.log :: GET /api/database/3 500 23.7 ms (4 DB calls)
{:via [{:type java.lang.NumberFormatException, :message "For input string: \"*/5\"", :at [java.lang.NumberFormatException forInputString "NumberFormatException.java" 65]}],
:trace
[[java.lang.NumberFormatException forInputString "NumberFormatException.java" 65]
[java.lang.Integer parseInt "Integer.java" 638]
[java.lang.Integer parseInt "Integer.java" 770]
[metabase.util.cron$cron__GT_digit invokeStatic "cron.clj" 124]
[metabase.util.cron$cron__GT_digit invoke "cron.clj" 121]
[metabase.util.cron$eval37331$cron_string__GT_schedule_map__37336$fn__37337 invoke "cron.clj" 143]
[metabase.util.cron$eval37331$cron_string__GT_schedule_map__37336 invoke "cron.clj" 139]
[metabase.api.database$eval98016$expanded_schedules__98021$fn__98022 invoke "database.clj" 223]
[metabase.api.database$eval98016$expanded_schedules__98021 invoke "database.clj" 221]
[metabase.api.database$add_expanded_schedules invokeStatic "database.clj" 229]
[metabase.api.database$add_expanded_schedules invoke "database.clj" 225]
[metabase.api.database$fn__98049 invokeStatic "database.clj" 257]
[metabase.api.database$fn__98049 invoke "database.clj" 250]
[compojure.core$wrap_response$fn__33282 invoke "core.clj" 160]
[compojure.core$wrap_route_middleware$fn__33266 invoke "core.clj" 132]
[compojure.core$wrap_route_info$fn__33271 invoke "core.clj" 139]
[compojure.core$wrap_route_matches$fn__33275 invoke "core.clj" 151]
[compojure.core$routes$fn__33294$f__33295 invoke "core.clj" 198]
[compojure.core$routes$fn__33294$f__33295$respond_SINGLEQUOTE___33296 invoke "core.clj" 197]
[compojure.core$wrap_route_matches$fn__33275 invoke "core.clj" 153]
[compojure.core$routes$fn__33294$f__33295 invoke "core.clj" 198]
[compojure.core$routes$fn__33294$f__33295$respond_SINGLEQUOTE___33296 invoke "core.clj" 197]
[compojure.core$wrap_route_matches$fn__33275 invoke "core.clj" 152]
[compojure.core$routes$fn__33294$f__33295 invoke "core.clj" 198]
[compojure.core$routes$fn__33294$f__33295$respond_SINGLEQUOTE___33296 invoke "core.clj" 197]
[compojure.core$wrap_route_matches$fn__33275 invoke "core.clj" 152]
[compojure.core$routes$fn__33294$f__33295 invoke "core.clj" 198]
[compojure.core$routes$fn__33294$f__33295$respond_SINGLEQUOTE___33296 invoke "core.clj" 197]
[compojure.core$wrap_route_matches$fn__33275 invoke "core.clj" 152]
[compojure.core$routes$fn__33294$f__33295 invoke "core.clj" 198]
[compojure.core$routes$fn__33294$f__33295$respond_SINGLEQUOTE___33296 invoke "core.clj" 197]
[compojure.core$wrap_route_matches$fn__33275 invoke "core.clj" 152]
[compojure.core$routes$fn__33294$f__33295 invoke "core.clj" 198]
[compojure.core$routes$fn__33294$f__33295$respond_SINGLEQUOTE___33296 invoke "core.clj" 197]
[compojure.core$wrap_route_matches$fn__33275 invoke "core.clj" 153]
[compojure.core$routes$fn__33294$f__33295 invoke "core.clj" 198]
[compojure.core$routes$fn__33294$f__33295$respond_SINGLEQUOTE___33296 invoke "core.clj" 197]
[compojure.core$wrap_route_matches$fn__33275 invoke "core.clj" 152]
[compojure.core$routes$fn__33294$f__33295 invoke "core.clj" 198]
[compojure.core$routes$fn__33294$f__33295$respond_SINGLEQUOTE___33296 invoke "core.clj" 197]
[compojure.core$wrap_route_matches$fn__33275 invoke "core.clj" 152]
[compojure.core$routes$fn__33294$f__33295 invoke "core.clj" 198]
[compojure.core$routes$fn__33294$f__33295$respond_SINGLEQUOTE___33296 invoke "core.clj" 197]
[compojure.core$wrap_route_matches$fn__33275 invoke "core.clj" 152]
[compojure.core$routes$fn__33294$f__33295 invoke "core.clj" 198]
[compojure.core$routes$fn__33294$f__33295$respond_SINGLEQUOTE___33296 invoke "core.clj" 197]
[compojure.core$wrap_route_matches$fn__33275 invoke "core.clj" 152]
[compojure.core$routes$fn__33294$f__33295 invoke "core.clj" 198]
[compojure.core$routes$fn__33294$f__33295$respond_SINGLEQUOTE___33296 invoke "core.clj" 197]
[compojure.core$wrap_route_matches$fn__33275 invoke "core.clj" 153]
[compojure.core$routes$fn__33294$f__33295 invoke "core.clj" 198]
[compojure.core$routes$fn__33294$f__33295$respond_SINGLEQUOTE___33296 invoke "core.clj" 197]
[compojure.core$wrap_route_matches$fn__33275 invoke "core.clj" 152]
[compojure.core$routes$fn__33294$f__33295 invoke "core.clj" 198]
[compojure.core$routes$fn__33294$f__33295$respond_SINGLEQUOTE___33296 invoke "core.clj" 197]
[compojure.core$wrap_route_matches$fn__33275 invoke "core.clj" 153]
[compojure.core$routes$fn__33294$f__33295 invoke "core.clj" 198]
[compojure.core$routes$fn__33294$f__33295$respond_SINGLEQUOTE___33296 invoke "core.clj" 197]
[compojure.core$wrap_route_matches$fn__33275 invoke "core.clj" 152]
[compojure.core$routes$fn__33294$f__33295 invoke "core.clj" 198]
[compojure.core$routes$fn__33294$f__33295$respond_SINGLEQUOTE___33296 invoke "core.clj" 197]
[compojure.core$wrap_route_matches$fn__33275 invoke "core.clj" 153]
[compojure.core$routes$fn__33294$f__33295 invoke "core.clj" 198]
[compojure.core$routes$fn__33294$f__33295$respond_SINGLEQUOTE___33296 invoke "core.clj" 197]
[compojure.core$wrap_route_matches$fn__33275 invoke "core.clj" 153]
[compojure.core$routes$fn__33294$f__33295 invoke "core.clj" 198]
[compojure.core$routes$fn__33294$f__33295$respond_SINGLEQUOTE___33296 invoke "core.clj" 197]
[compojure.core$wrap_route_matches$fn__33275 invoke "core.clj" 153]
[compojure.core$routes$fn__33294$f__33295 invoke "core.clj" 198]
[compojure.core$routes$fn__33294$f__33295$respond_SINGLEQUOTE___33296 invoke "core.clj" 197]
[compojure.core$wrap_route_matches$fn__33275 invoke "core.clj" 153]
[compojure.core$routes$fn__33294$f__33295 invoke "core.clj" 198]
[compojure.core$routes$fn__33294$f__33295$respond_SINGLEQUOTE___33296 invoke "core.clj" 197]
[compojure.core$wrap_route_matches$fn__33275 invoke "core.clj" 153]
[compojure.core$routes$fn__33294$f__33295 invoke "core.clj" 198]
[compojure.core$routes$fn__33294 invoke "core.clj" 200]
[metabase.server.middleware.auth$enforce_authentication$fn__90616 invoke "auth.clj" 14]
[compojure.core$routes$fn__33294$f__33295 invoke "core.clj" 198]
[compojure.core$routes$fn__33294 invoke "core.clj" 200]
[compojure.core$make_context$handler__33362 invoke "core.clj" 287]
[compojure.core$make_context$fn__33364 invoke "core.clj" 296]
[compojure.core$routes$fn__33294$f__33295 invoke "core.clj" 198]
[compojure.core$routes$fn__33294$f__33295$respond_SINGLEQUOTE___33296 invoke "core.clj" 197]
[compojure.core$make_context$fn__33364 invoke "core.clj" 297]
[compojure.core$routes$fn__33294$f__33295 invoke "core.clj" 198]
[compojure.core$routes$fn__33294$f__33295$respond_SINGLEQUOTE___33296 invoke "core.clj" 197]
[compojure.core$make_context$fn__33364 invoke "core.clj" 297]
[compojure.core$routes$fn__33294$f__33295 invoke "core.clj" 198]
[compojure.core$routes$fn__33294$f__33295$respond_SINGLEQUOTE___33296 invoke "core.clj" 197]
[compojure.core$make_context$fn__33364 invoke "core.clj" 297]
[compojure.core$routes$fn__33294$f__33295 invoke "core.clj" 198]
[compojure.core$routes$fn__33294$f__33295$respond_SINGLEQUOTE___33296 invoke "core.clj" 197]
[compojure.core$make_context$fn__33364 invoke "core.clj" 297]
[compojure.core$routes$fn__33294$f__33295 invoke "core.clj" 198]
[compojure.core$routes$fn__33294$f__33295$respond_SINGLEQUOTE___33296 invoke "core.clj" 197]
[compojure.core$make_context$fn__33364 invoke "core.clj" 297]
[compojure.core$routes$fn__33294$f__33295 invoke "core.clj" 198]
[compojure.core$routes$fn__33294$f__33295$respond_SINGLEQUOTE___33296 invoke "core.clj" 197]
[compojure.core$make_context$fn__33364 invoke "core.clj" 297]
[compojure.core$routes$fn__33294$f__33295 invoke "core.clj" 198]
[compojure.core$routes$fn__33294$f__33295$respond_SINGLEQUOTE___33296 invoke "core.clj" 197]
[metabase.api.routes$fn__103931$fn__103933 invoke "routes.clj" 69]
[compojure.core$routes$fn__33294$f__33295 invoke "core.clj" 198]
[compojure.core$routes$fn__33294 invoke "core.clj" 200]
[clojure.lang.AFn applyToHelper "AFn.java" 160]
[clojure.lang.AFn applyTo "AFn.java" 144]
[clojure.core$apply invokeStatic "core.clj" 665]
[clojure.core$apply invoke "core.clj" 660]
[metabase.server.routes$fn__104081$fn__104082 doInvoke "routes.clj" 57]
[clojure.lang.RestFn invoke "RestFn.java" 436]
[compojure.core$routes$fn__33294$f__33295 invoke "core.clj" 198]
[compojure.core$routes$fn__33294 invoke "core.clj" 200]
[compojure.core$make_context$handler__33362 invoke "core.clj" 287]
[compojure.core$make_context$fn__33364 invoke "core.clj" 296]
[compojure.core$routes$fn__33294$f__33295 invoke "core.clj" 198]
[compojure.core$routes$fn__33294$f__33295$respond_SINGLEQUOTE___33296 invoke "core.clj" 197]
[compojure.core$wrap_route_matches$fn__33275 invoke "core.clj" 152]
[compojure.core$routes$fn__33294$f__33295 invoke "core.clj" 198]
[compojure.core$routes$fn__33294$f__33295$respond_SINGLEQUOTE___33296 invoke "core.clj" 197]
[compojure.core$wrap_route_matches$fn__33275 invoke "core.clj" 152]
[compojure.core$routes$fn__33294$f__33295 invoke "core.clj" 198]
[compojure.core$routes$fn__33294$f__33295$respond_SINGLEQUOTE___33296 invoke "core.clj" 197]
[compojure.core$wrap_route_matches$fn__33275 invoke "core.clj" 152]
[compojure.core$routes$fn__33294$f__33295 invoke "core.clj" 198]
[compojure.core$routes$fn__33294$f__33295$respond_SINGLEQUOTE___33296 invoke "core.clj" 197]
[metabase.server.routes$fn__104069$fn__104071 invoke "routes.clj" 41]
[compojure.core$routes$fn__33294$f__33295 invoke "core.clj" 198]
[compojure.core$routes$fn__33294 invoke "core.clj" 200]
[metabase.server.middleware.exceptions$catch_uncaught_exceptions$fn__90750 invoke "exceptions.clj" 96]
[metabase.server.middleware.exceptions$catch_api_exceptions$fn__90747 invoke "exceptions.clj" 84]
[metabase.server.middleware.log$log_api_call$fn__91165$fn__91166 invoke "log.clj" 195]
[toucan.db$_do_with_call_counting invokeStatic "db.clj" 216]
[toucan.db$_do_with_call_counting invoke "db.clj" 209]
[metabase.server.middleware.log$log_api_call$fn__91165 invoke "log.clj" 189]
[metabase.server.middleware.security$add_security_headers$fn__90705 invoke "security.clj" 142]
[metabase.server.middleware.json$wrap_json_body$fn__90938 invoke "json.clj" 62]
[metabase.server.middleware.json$wrap_streamed_json_response$fn__90956 invoke "json.clj" 98]
[ring.middleware.keyword_params$wrap_keyword_params$fn__104384 invoke "keyword_params.clj" 55]
[ring.middleware.params$wrap_params$fn__104408 invoke "params.clj" 69]
[metabase.server.middleware.misc$maybe_set_site_url$fn__47584 invoke "misc.clj" 58]
[metabase.server.middleware.session$bind_current_user$fn__56162$fn__56163 invoke "session.clj" 277]
[metabase.server.middleware.session$do_with_current_user invokeStatic "session.clj" 258]
[metabase.server.middleware.session$do_with_current_user invoke "session.clj" 250]
[metabase.server.middleware.session$bind_current_user$fn__56162 invoke "session.clj" 276]
[metabase.server.middleware.session$wrap_current_user_info$fn__56145 invoke "session.clj" 236]
[metabase.server.middleware.session$wrap_session_id$fn__56131 invoke "session.clj" 182]
[metabase.server.middleware.auth$wrap_api_key$fn__90624 invoke "auth.clj" 27]
[ring.middleware.cookies$wrap_cookies$fn__104288 invoke "cookies.clj" 216]
[metabase.server.middleware.misc$add_content_type$fn__47567 invoke "misc.clj" 27]
[metabase.server.middleware.misc$disable_streaming_buffering$fn__47592 invoke "misc.clj" 75]
[ring.middleware.gzip$wrap_gzip$fn__104338 invoke "gzip.clj" 86]
[metabase.server.middleware.misc$bind_request$fn__47595 invoke "misc.clj" 92]
[metabase.server.middleware.ssl$redirect_to_https_middleware$fn__91190 invoke "ssl.clj" 38]
[clojure.lang.Var invoke "Var.java" 393]
[metabase.server$async_proxy_handler$fn__90569 invoke "server.clj" 71]
[metabase.server.proxy$org.eclipse.jetty.server.handler.AbstractHandler$ff19274a handle nil -1]
[org.eclipse.jetty.server.handler.HandlerWrapper handle "HandlerWrapper.java" 127]
[org.eclipse.jetty.server.Server handle "Server.java" 516]
[org.eclipse.jetty.server.HttpChannel lambda$handle$1 "HttpChannel.java" 383]
[org.eclipse.jetty.server.HttpChannel dispatch "HttpChannel.java" 556]
[org.eclipse.jetty.server.HttpChannel handle "HttpChannel.java" 375]
[org.eclipse.jetty.server.HttpConnection onFillable "HttpConnection.java" 273]
[org.eclipse.jetty.io.AbstractConnection$ReadCallback succeeded "AbstractConnection.java" 311]
[org.eclipse.jetty.io.FillInterest fillable "FillInterest.java" 105]
[org.eclipse.jetty.io.ChannelEndPoint$1 run "ChannelEndPoint.java" 104]
[org.eclipse.jetty.util.thread.strategy.EatWhatYouKill runTask "EatWhatYouKill.java" 336]
[org.eclipse.jetty.util.thread.strategy.EatWhatYouKill doProduce "EatWhatYouKill.java" 313]
[org.eclipse.jetty.util.thread.strategy.EatWhatYouKill tryProduce "EatWhatYouKill.java" 171]
[org.eclipse.jetty.util.thread.strategy.EatWhatYouKill run "EatWhatYouKill.java" 129]
[org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread run "ReservedThreadExecutor.java" 375]
[org.eclipse.jetty.util.thread.QueuedThreadPool runJob "QueuedThreadPool.java" 773]
[org.eclipse.jetty.util.thread.QueuedThreadPool$Runner run "QueuedThreadPool.java" 905]
[java.lang.Thread run "Thread.java" 834]],
:cause "For input string: \"*/5\"",
:message "For input string: \"*/5\"",
:type java.lang.NumberFormatException}
```

Tested against `release-x.38.x` branch.

Contributor guide

Open the contributing guide

Research direction

Start in metabase.util.cron in cron.clj around lines 121-143, then trace how metabase.api.database uses the parsed schedule around database.clj lines 221-229. Reproduce the failure with the reported `0 */5 * * * ? *` schedule and GET /api/database/3. Done means the valid interval schedule no longer causes a server error.

Written by the indexing model from the issue text.

Assessment

Tech stack
clojure
Domain
api, backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.