apache / apache/doris

[Discussion] Multi-Region Data Locality + Federated Execution

Open
#66,645 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
15.9k
Forks
3.9k
Avg merge
2d 23h
Merged PRs (30d)
520

Description

Hi team,

I am looking into guarantee where compute happening in region, result to FE, no raw data crosses borders and user should be able to see aggregated result.
Please let me know if we have some work/discussion already done like this :

```
Client


FE Leader ──── Parses SQL → generates Logical Plan → Physical Query Plan

│ PQP = DAG of fragments, each assigned to specific BE/CN nodes

├─► Fragment 1 (Scan + partial agg) ──► BE/CN-A [data local]
├─► Fragment 2 (Scan + partial agg) ──► BE/CN-B [data local]
├─► Fragment 3 (Scan + partial agg) ──► BE/CN-C [data local]
│ │ │ │
│ └──────────────┴──────────────┘
│ Exchange nodes (HASH / BROADCAST / UNPARTITIONED)
│ │
└─► Final Fragment (merge agg, top-N, ORDER BY) ──► single BE or FE itself

Result Set ──► back to FE ──► Client
```

Contributor guide

Open the contributing guide

Research direction

No files, tests, or concrete entry points are named. Start by reviewing how the FE leader, BE/CN nodes, logical plans, physical query plans, fragments, and exchange nodes currently work. Before implementation, the discussion needs a bounded proposal with affected components and acceptance criteria for regional execution and data locality.

Written by the indexing model from the issue text.

Assessment

Tech stack
sql
Domain
backend, databases, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.