[Discuss] Doris Roadmap 2026

# _"Scale Intelligence, Accelerate Insights"_

Building on 2025's achievements in vector search and indexing capabilities, Apache Doris continues to deepen its AI support in 2026. This roadmap focuses on advancing **AI & Hybrid Search** capabilities while enhancing query performance, storage efficiency, and data lake integration.

**AI & Hybrid Search Innovation:**
- Scale vector index to support **10 billion vectors** per table with disk-based ANN
- Enhance full-text search with query expressions, scoring, and multi-index support
- Extend hybrid search to **Iceberg** for unified analytics

**Core Enhancements:**
- Query engine optimization for complex data types and ETL processing
- Storage improvements for ultra-large tablets and compute-storage separation
- Data lake integration with Iceberg V3 and Paimon support

[Roadmap 2025](https://github.com/apache/doris/issues/47948)
[Roadmap 2024](https://github.com/apache/doris/issues/30669)
[Roadmap 2023](https://github.com/apache/doris/issues/16392)
[Roadmap 2022](https://github.com/apache/doris/issues/7502)

# AI & Hybrid Search

## Vector Index

- [x] Implement index-only scan for vector index (#57243)
- [x] Implement disk-based ANN (Approximate Nearest Neighbor) for vector index (#61160)
- [ ] Optimize compaction policy for vector index
- [ ] Enhance vector index capability to support 10 billion vectors in a single table (#61986)
- [ ] Introduce vector index support for Iceberg tables
- [x] Support vector index in merge on write engine. (#61994)

## Full-Text Search

- [x] Introduce more query expressions: query string and Boolean query (#58545)
- [x] Implement scoring functionality in the text index (#59847)
- [x] Introduce multi-index support for a single column (#59117)
- [ ] Add text index support for Iceberg tables
- [ ] Integrate scoring with global lazy materialization

# Query Engine

## Performance

- [x] Optimize column pruning for complex data types (struct, array, map) (#57195, #57204)
- [x] Optimize expression execution for cases such as `CASE WHEN` and non-const `LIKE` (#57025)
- [x] Enhance partition pruning capability (#62589)
- [ ] Optimize broadcast join performance
- [ ] Implement query condition cache functionality
- [x] Enhance zonemap evaluation to support expressions (#63389)

## ETL/Incremental Processing

- [x] Enhance spill-to-disk capability to support TPC-DS 10TB workload using 16GB memory (#61212)
- [x] Implement `MERGE INTO` statement (#57044, #60482)
- [ ] Implement binlog and incremental materialized view functionality (#65418)
- [ ] Implement global query buffer management to reduce memory usage for single queries and make query usage more predictable
- [x] Implement progress bar for long-running queries (#60567)

## New Features

- [x] Implement UNNEST functionality (#58228)
- [x] Implement recursive CTE (Common Table Expression) (#58916)
- [x] Implement ASOF join functionality (#59591)
- [x] Introduce Python UDF (User-Defined Function) support (#59543)
- [x] Introduce nested variant data type support (#60847)
- [ ] Enhance function compatibility with Snowflake (#60982)

## New DataTypes

- [x] Introduce timestamp with timezone data type (#59399)
- [ ] Introduce binary data type

## Enhancement

- [ ] Unify predicate and expression framework between external tables and internal tables
- [x] Implement short-circuit expression evaluation (#59830)
- [x] Unify local exchange and global exchange, and move local exchange to FE planner (#63366)

# Data Storage

## Storage Format

- [ ] Optimize compression ratio for string data
- [ ] Enhance storage format to support 10k columns in a single file
- [ ] Optimize column metadata management for random access
- [ ] Optimize nullable column read performance
- [ ] Optimize storage for sparse columns in variant data type (#61539)
- [ ] Implement partial update functionality for variant sub-fields (#63182)

## Data management

- [ ] Enhance tablet management to support ultra-large tablets (100GB+)
- [ ] Optimize MOW (Merge-On-Write) import performance for large tablets

## File Cache

- [x] Implement table-level cross-compute group synchronized preheating (#63832)
- [ ] Implement partition time-based TTL (Time-To-Live) support
- [ ] Enhance SQL query capability for more granular and reliable cache usage statistics
- [ ] Optimize diskless/slow disk scenarios to prevent local disk from becoming a file cache throughput bottleneck
- [ ] Implement cache block/allow list policy for fine-grade cache management. (#59065)

## Compute-Storage Separation

- [ ] Implement ultra-fast elastic balance scheduling
- [x] Enhance read-write separation: bind compaction to write compute groups (#60310)
- [ ] Implement distributed cache support for sharing cache across multiple compute groups
- [ ] Enhance persistent metadata caching to reduce dependency on metadata service and improve performance

### Data Import

- [ ] Optimize memory management for large imports with many active tablets that may result in many small files: implement memtable disk spill
- [ ] Optimize memory control for scenarios with very large single-row single-column data
- [x] Introduce support for more data import sources, such as AWS Kinesis (#61325)

# Data Lakes

## Lake Format Performance

- [x] Implement Parquet format Page Cache capability (#59307)
- [ ] Enable Data Cache by default
- [x] Enhance metadata parsing, planning, and caching for ultra-large scale Iceberg and Paimon (#59716, #60478)
- [x] Implement Condition Cache for Iceberg and Paimon (#60897)

## Materialized View

- [ ] Implement snapshot-level incremental refresh for materialized views based on Iceberg and Paimon
- [ ] Implement materialized view construction based on Paimon and Iceberg

## Data interoperability

- [x] Implement comprehensive Iceberg V3 support (#59272, #61398)
- [x] Implement Iceberg data sorting functionality (#60540)
- [x] Implement Iceberg Data Rewrite functionality (#56413)
- [x] Implement Iceberg Delete/Update functionality (#60482)
- [ ] Implement Iceberg/Parquet Variant data type support (#63192)
- [ ] Implement Paimon data write (#56005)
- [ ] Implement native reader for Paimon MOR (Merge-On-Read) tables (#56005)
- [ ] Implement Fluss integration (#59220)
- [ ] Implement Paimon Vector and Blob data type support
- [ ] Implement standardized Arrow Flight Data Catalog (#59270)

## Metadata Interoperability

- [ ] Implement unified permission management for Iceberg REST Catalog (#63068)
- [ ] Implement integration with third-party authentication and authorization systems
- [ ] Implement Open Metadata API

## Connector Extensibility

- [ ] Decouple built-in connectors from FE core int loadable plugins (#65185)

# Security

- [x] Enhance object storage support for IAM role-based authentication from more cloud vendors (#60498)

# Others

- [ ] Refactor all third-party builds to use CMake
- [ ] Implement hermetic build support