metabase / metabase/metabase

MySQL user-variables are not reset on concurrent queries

Open
#6,550 1 comment 4 reactions 0 assignees View on GitHub
.Limitation Database/MySQL Querying/ Type:New Feature
Dominant language
Clojure
Stars
49.3k
Forks
6.8k
Avg merge
1d 13h
Merged PRs (30d)
653

Description

Hello,

I use a SQL request for calculating the Cumulative Total. In MySQL 5.7 you can calculate the Cumulative Total ONLY with using variables in request.

Request example:

```
SELECT
TIME
,PL AS PL
,@PL_TOTAL := (IFNULL(@PL_TOTAL, 0) + PL) AS TOTAL_PL
FROM
(
SELECT
MT4_TRADES_ONLINE.TIME
,SUM(MT4_TRADES_ONLINE.PL * r.Rate / MT4_USERS.DELIMER) AS PL
FROM
MT4_BALANCE MT4_TRADES_ONLINE
LEFT JOIN MT4_USERS_MEMORY MT4_USERS USING(LOGIN, REPORT_ID)
LEFT JOIN ConvertationRate r ON r.Base = MT4_USERS.CURRENCY AND r.Quote = 'USD'
WHERE
MT4_TRADES_ONLINE.TIMEFRAME = {{TIMEFRAME}}
[[AND {{TIME}}]]
[[AND MT4_USERS.BUSINESS_TYPE_ACCOUNT RLIKE {{TYPE_ACCOUNT}}]]
[[AND MT4_TRADES_ONLINE.RATIO RLIKE {{RATIOS}}]]
[[AND MT4_TRADES_ONLINE.LOGIN RLIKE {{LOGINS}}]]
[[AND MT4_USERS.NAME RLIKE {{NAME}}]]
[[AND {{REGDATE}}]]
[[AND MT4_USERS.ZIPCODE RLIKE {{ZIPCODE}}]]
[[AND {{IS_SWAPFREE}}]]
[[AND {{COUNTRY}}]]
[[AND MT4_USERS.COMMENT RLIKE {{COMMENT}}]]
GROUP BY
TIME
[[HAVING ABS(PL) >= {{HAVING}}]]
ORDER BY
TIME
)
as PL
```

MySQL keeps variables values during one session. Metabase uses same session for all request to database. Thereby my variable for the Cumulative Total @PL_TOTAL doesn’t reset it’s value to NULL after the first request is completed. When I run second request, the variable @PL_TOTAL still contains old, not NULL value. And it’s new value sums with it’s old value! Thereby everytime when I reload my dashboard the Cumulative Total’s value gets bigger and bigger.
Watch this video: http://recordit.co/kCyIYlS9pB

If you want to avoid of this problem, you have to “hardcode” an UNION ALL constraction with combination with HAVING.

So there is the example of this modification of this request:

```
SELECT
TIME
,PL AS PL
,@PL_TOTAL := (IFNULL(@PL_TOTAL, 0) + PL) AS TOTAL_PL
FROM
(
(
SELECT
NULL TIME
,@PL_TOTAL := 0 PL
)
UNION ALL
(
SELECT
MT4_TRADES_ONLINE.TIME
,SUM(MT4_TRADES_ONLINE.PL * r.Rate / MT4_USERS.DELIMER) AS PL
FROM
MT4_BALANCE MT4_TRADES_ONLINE
LEFT JOIN MT4_USERS_MEMORY MT4_USERS USING(LOGIN, REPORT_ID)
LEFT JOIN ConvertationRate r ON r.Base = MT4_USERS.CURRENCY AND r.Quote = 'USD'
WHERE
MT4_TRADES_ONLINE.TIMEFRAME = {{TIMEFRAME}}
[[AND {{TIME}}]]
[[AND MT4_USERS.BUSINESS_TYPE_ACCOUNT RLIKE {{TYPE_ACCOUNT}}]]
[[AND MT4_TRADES_ONLINE.RATIO RLIKE {{RATIOS}}]]
[[AND MT4_TRADES_ONLINE.LOGIN RLIKE {{LOGINS}}]]
[[AND MT4_USERS.NAME RLIKE {{NAME}}]]
[[AND {{REGDATE}}]]
[[AND MT4_USERS.ZIPCODE RLIKE {{ZIPCODE}}]]
[[AND {{IS_SWAPFREE}}]]
[[AND {{COUNTRY}}]]
[[AND MT4_USERS.COMMENT RLIKE {{COMMENT}}]]
GROUP BY
TIME
[[HAVING ABS(PL) >= {{HAVING}}]]
ORDER BY
TIME
)
)
as PL
HAVING
TIME IS NOT NULL
```

This modification makes the request too ugly and heavy.
Are there any solutions which allows me to use MySQL variables?

Contributor guide

Open the contributing guide

Research direction

Start with the MySQL session behavior described in the issue and trace how Metabase reuses database sessions for concurrent queries. Reproduce the cumulative-total query across repeated dashboard loads, then identify the connection or session boundary involved. Done means user variables no longer leak between requests or concurrent queries, with the behavior verified against the reported example.

Written by the indexing model from the issue text.

Assessment

Tech stack
mysql
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.