influxdata / influxdata/telegraf
Auto create table in BigQuery
- Dominant language
- Go
- Stars
- 17.8k
- Forks
- 5.8k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 161
Description
### Use Case
With some input plugins the metric names and formats are unknown on beforehand, the number of tags and fields can vary over time. Therefore it would be nice if the `outputs.bigquery` plugin could handle this in a way similar to the `outputs.sql` plugin.
### Expected behavior
Telegraf creates the required table in BigQuery with all the corresponding metric fields and tags just as the sql
### Actual behavior
Tables for metrics need to be created on beforehand in BigQuery.
### Additional info
[Here is a PoC](https://gist.github.com/Hipska/1357083d666c5f57e2e0271159fab943) which adds a new BigQuery field on the fly, proving it should be possible.
Contributor guide
Research direction
Review the outputs.bigquery plugin alongside outputs.sql and the linked PoC to understand how metric fields and tags should map to table creation. Verify completion with unknown metric schemas by confirming that the required BigQuery tables are created automatically.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, google-cloud
- Domain
- cloud, databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100