influxdata / influxdata/ui

[Notebooks Enhancement] Improve intuitiveness and performance of schema exploration

Open
#384 7 comments 0 reactions 0 assignees View on GitHub
team/design
Dominant language
TypeScript
Stars
117
Forks
51
Avg merge
2d 15h
Merged PRs (30d)
4

Description

@samhld commented on [Wed Dec 02 2020](https://github.com/influxdata/influxdb/issues/20239)

__Current behavior:__
The data exploration ("Select a metric") cell type defaults to showing you a comprehensive list of all possible column values. This looks like the below gif (compared with the experience in the original schema explorer). Idk why it came out so slow:

![books](https://user-images.githubusercontent.com/4852912/100929479-8ff9fe00-349c-11eb-9c06-01e45e59c0f9.gif)

__Problem:__
1) Results seem to be slower...which I would expect given the comprehensiveness of the query/ies underlying this functionality.
2) It has lag. Again, the result set could be too large.
3) It's flat. The list is too long to grok and doesn't make intuitive sense given the InfluxDB data model (I come back to this in next section)

__What we know:__
The Influx data model follows a logical form similar to the below:
![Screen Shot 2020-10-30 at 1 23 29 PM](https://user-images.githubusercontent.com/4852912/100930069-6a212900-349d-11eb-88a0-75cc5e280210.png)

From the perspective of a user and their access patterns, a query logically filters first by Measurement and then begins to seek matching Tags/columns. Similar to a SQL query, a user first determines which Table they will need to find their desired data. A flat representation of this concept seems likely to confuse users.

The expectation of this feature is that the user knows which metrics they want to select coming into this.

If users don't know anything about the Influx data model, they will need assistance with knowing:
* Which metrics they have
* Which "tables" they live in
* Which pieces of metadata the metrics are associated with

__Proposal 1:__
Use the original Data Explorer for this cell. It is intuitive for users (I hear this a lot).

I know a known issue is that its underlying queries were expensive....but is that improved with this flat model? The above gif alludes to otherwise.

Also, I've heard talks about caching this information (which we absolutely should do) anyways.

__Proposal 2:__
Keep the basic idea of the current representation but manifest series keys as dot-notated keys that indicate relationships similar to the Line Protocol diagram above. Examples:

`disk.us-west.host01.sda0.used_percent`
`system.us-east.host1002.disk_used`

In the above series keys, the first line segment (top) is always the Measurement name. Every segment after that and _before_ the last are Tags and order does not matter (this is not a document-/directory-style relationship after all). The last segment is the Field/"metric". The Tags are less meaningful when doing data exploration so they can be "shortened" with ellipses or some other notation...if/when needed. Perhaps they could be omitted entirely (I'll think on that more).

The benefit of this is there is an implied relationship between the objects. I believe this makes more sense to a user whether or not they are familiar with the Influx data model. The information missing would be what the line segments mean (easily tool-tipped) but the current flat representations has the same problem manifested slightly differently...and again, more confusingly IMO.

One of the reasons for the current flat representation was that users don't necessarily know the difference between Measurements and Fields so we needed a way to display all options when someone filtered for, say `disk`. Does the user want a table with a bunch of `disk` metrics or simply a `disk` metric...and do they know that upfront? This dot-notation also resolves that issue by displaying all the options but with more context.

__Proposal 3:__
Don't display Tag values, just keys. This provides the user the ability to filter by Measurement name and other metadata. From there, they get a result set of Fields (values to the `_field` Tag, in this case). This is sort of a hybrid approach to the current state and the original Data Explorer depicted on the right side of the gif at the top of the issue.

It, along with the first two proposals, satisfies the bullets in the "What we know" section as well as resolves the issues pointed to earlier.

---

@rbetts commented on [Wed Dec 02 2020](https://github.com/influxdata/influxdb/issues/20239#issuecomment-737516404)

As we think about metadata presentation in flows, also need to remember that we need a system for arbitrary cardinality. In the future user with a billion series and 100,000 measurements should be able to discover, understand, and navigate their data.

---

@pauldix commented on [Wed Dec 02 2020](https://github.com/influxdata/influxdb/issues/20239#issuecomment-737518386)

Given the underlying design of IOx, I think the number of measurements will be significantly less than 100k. Likely a few thousand that the most, otherwise the user should maybe be rethinking their schema design. I'm guessing that having 100k measurements will result in poor performance (although who knows).

Tag value cardinalities will effectively be unbounded, but getting back all those unique values will take time. Generally, when looking for tag values, it should be scoped by some time period, which is flexible, and then use `LIMIT` and `OFFSET` to paginate through results.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.