influxdata / influxdata/influxdb

[v3] Strong Request: Reintroduce or Provide Compatibility for Flux Language in InfluxDB 3.0 (IOx)

Open
#26,989 1 comment 1 reaction 0 assignees View on GitHub
v3
Dominant language
Rust
Stars
31.7k
Forks
3.7k
Avg merge
13h 37m
Merged PRs (30d)
8

Description

# 📢 Strong Request: Reintroduce or Provide Compatibility for Flux Language in InfluxDB 3.0 (IOx)

Dear InfluxDB Team,

I am submitting this critical issue on behalf of community users who heavily rely on the unique capabilities of the **Flux language**. We strongly urge you to reconsider or implement a compatibility layer for Flux within the InfluxDB 3.0 (Arrow/DataFusion-based IOx) architecture.

---

### 📉 The Challenge: Loss of Elegant Time-Series Querying

While the migration to SQL and the IOx architecture brings commendable performance gains, the removal of Flux has resulted in a significant loss of the most elegant and efficient way to handle specific, high-frequency time-series query patterns.

### ✅ Key Arguments for Reintroducing Flux

The complexity introduced by the SQL-only approach directly impacts developer efficiency and system maintainability.

#### 1. Elegant Multi-Field Latest Value Querying is Gone

* **In Flux:** Obtaining the latest non-null value and its corresponding **independent timestamp** for multiple fields requires only the simple `|> last()` function.
* **In 3.0 (LVC):** The **LVC (`last_cache()`) solution**, while fast, requires **prior configuration and enabling**. This sacrifices the dynamic, flexible, and out-of-the-box advantage of Flux, unnecessarily adding maintenance overhead for what is a fundamental time-series query.

#### 2. Superiority in Complex Data Flow and ETL

* Flux’s pipeline (`|>`) operations are ideal for time-series **ETL, cross-measurement joins, customized data resampling, and complex aggregation**.
* Replicating this logic using standard SQL often necessitates verbose and error-prone **subqueries, CTEs, and complex window functions**, making the code less readable and harder to maintain than its Flux counterpart.

#### 3. Community Investment and Migration Costs

* The community has invested substantial time and resources into learning Flux and building mission-critical workflows.
* The deprecation forces users to abandon these existing codebases and knowledge assets, creating a **significant barrier and cost** for upgrading to InfluxDB 3.0.

---

### 💡 Proposed Solutions (Recommendations for the Roadmap)

We understand the engineering challenge of running the native Flux engine atop DataFusion. We propose the team evaluate the feasibility of the following alternatives to restore this vital functionality:

1. **Develop a Flux-to-SQL Compiler:** Offer a compatibility layer that efficiently translates common Flux patterns (e.g., `|> last()`, `|> group()`, `|> aggregateWindow()`) into highly optimized DataFusion SQL or Arrow native operations.
2. **Reintroduce Core Flux Functions as UDFs:** At the very least, re-introduce the behavior of Flux functions that are notoriously difficult to replicate elegantly in standard SQL (e.g., the true multi-series latest value behavior of `last()`), as specialized **Time-Series User-Defined Functions (UDFs)**.

We sincerely hope the InfluxDB team considers this critical community feedback and includes a path for feature parity or function migration for existing Flux users in the future roadmap.

Thank you for your time and attention to this matter.

Contributor guide

Open the contributing guide

Research direction

No files, tests, or entry points are named. Start by reviewing the InfluxDB 3.0 IOx/DataFusion query architecture and its current SQL and Flux compatibility status; done would require a scoped compatibility design or roadmap decision covering the proposed alternatives.

Written by the indexing model from the issue text.

Assessment

Tech stack
sql
Domain
data-engineering, databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.