4paradigm / 4paradigm/OpenMLDB

count(*) over table in offline mode

Open
#3,652 0 comments 0 reactions 1 assignee Claimed by @tobegit3hub View on GitHub
batch-engine enhancement high-priority
Dominant language
C++
Stars
1.7k
Forks
331
Avg merge
12d 12h
Merged PRs (30d)
1

Description

We don't support agg over table in offline mode, only online mode supports.
So if user wants to get offline data count, it's hard to do. Now user can run it on spark-shell/pyspark.
But if user can't prepare the spark env, it's better to enable sparksql in openmldb.

pass --spark_conf with option `spark.openmldb.sparksql`

P.S. user should know that not all openmldb sql can be run on spark engine.

Contributor guide

Open the contributing guide

Research direction

Look for the offline mode execution path and the SQL parser. The issue mentions enabling sparksql via a spark.conf flag `spark.openmldb.sparksql`. Find where offline queries are routed and how to conditionally use Spark's SQL engine for aggregate functions like count(*). Check test files for offline query execution to understand the expected behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
spark, sql
Domain
backend, databases, machine-learning
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.