influxdata / influxdata/influxdb

Chronograf coupled to InfluxDB v2 does not display all buckets/DRBPs

Open
#23,362 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
31.7k
Forks
3.7k
Avg merge
13h 37m
Merged PRs (30d)
8

Description

Steps to reproduce:
List the minimal actions needed to reproduce the behavior.

  1. Create more than 18 buckets in InfluxDB2
  2. Run the Flux query
buckets()
  |> rename(columns: {"name": "_value"})
  |> keep(columns: ["_value"])
  1. Give each bucket you just created a Chronograf-compatible DBRP
  2. In the Chronograf UI, run the metaquery SHOW DATABASES -- you will see all the databases-from-buckets
  3. Also in Chronograf, run the same Flux query:
buckets()
  |> rename(columns: {"name": "_value"})
  |> keep(columns: ["_value"])

-- you will only see 18 buckets.

Expected behavior:
I expected all three of the queries to find all of my buckets/mapped-DBRPs.

Actual behavior:
The Chronograf query, which presumably goes through the Influx v1 compatibility API, paginates the buckets, and there's no way to request the rest of the pages, as far as I can tell.

Also even though the default page size is 20, I'm only getting 18 buckets back. I have a guess about that below.

Environment info:

  • System info: Linux 5.4.170+ x86_64
  • InfluxDB version: InfluxDB v2.2.0 (git: a2f8538837) build_date: 2022-04-06T17:36:40Z
  • Other relevant environment details: Installed from the helm chart influxdb2 2.0.12 / Chronograf 1.2.5, running at GKE. Chronograf image tag is 1.9.4.

Config:
Pretty much stock. I'm using my own ingress that gives me access to chronograf on a subpath of the influxdb2 FQDN and Influx and Chronograf are sharing an admin token from the same preexisting secret.

Given that I just wrote https://github.com/lsst-sqre/influx-bucket-mapper I can tell you that https://github.com/lsst-sqre/influx-bucket-mapper/blob/8385fd6fc168f1f137bd13acb9a44ff06a2216f4/bucketmapper#L153-L177 points to the fact that you only get 18 results from /api/v2/buckets.

Now that I've thought about it, the comment in that chunk of code is wrong: I do have an idea why, and that is that you're filtering the system buckets after pagination, and there are two of them at the top of the list, so the first page would only have 18 buckets.

I also would have expected the next page to be in links.next in the response to that query, but it's not, so I resorted to using after until I was getting an empty bucket list back. This seems clunky, and of course the operational problem is that there's no way to do either of those with buckets() that Chronograf Flux uses, so you're stuck with the first 18.

All I really want to do is get a full list of buckets via Chronograf (why Chronograf? Because InfluxDB v2 still doesn't support OAuth, which I need; add that and then I could dispense with Chronograf), and sort them in alphabetical, rather than creation, order. I don't think there's currently any way to do that.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the issue with more than 18 buckets, the /api/v2/buckets endpoint, and the Chronograf buckets() query. Trace how pagination and system-bucket filtering reach Chronograf, then verify that all mapped buckets are returned and can be ordered alphabetically.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
api, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.