Make the plan more readable and less verbose
- Dominant language
- Python
- Stars
- 7.3k
- Forks
- 1.4k
- Avg merge
- 3d 16h
- Merged PRs (30d)
- 15
Description
**Is your feature request related to a problem? Please describe.**
The `plan` is currently not much use other than to validate that the objects in the code base are correctly configured. This is because the plan output is so verbose that it is not very human readable. It contains a mass of irrelevant information and as a result we do not use it as the output is too large to post to Slack for example.
There is also no HTML rendering of the plan which could be useful to email to project owners prior to a release being deployed.
For example, removing a single `Field` object on the quickstart tutorial results in the following plan:
```
Updated feature view driver_hourly_stats_fresh
features: [name: "conv_rate"
value_type: FLOAT
, name: "acc_rate"
value_type: FLOAT
, name: "avg_daily_trips"
value_type: INT64
] -> [name: "conv_rate"
value_type: FLOAT
, name: "avg_daily_trips"
value_type: INT64
]
Updated on demand feature view transformed_conv_rate_fresh
sources: {'driver_hourly_stats_fresh': feature_view_projection {
feature_view_name: "driver_hourly_stats_fresh"
feature_columns {
name: "conv_rate"
value_type: FLOAT
}
feature_columns {
name: "acc_rate"
value_type: FLOAT
}
feature_columns {
name: "avg_daily_trips"
value_type: INT64
}
timestamp_field: "created"
created_timestamp_column: "created"
batch_source {
type: BATCH_FILE
timestamp_field: "event_timestamp"
created_timestamp_column: "created"
file_options {
uri: "data/driver_stats.parquet"
}
name: "driver_hourly_stats_source"
}
}
, 'vals_to_add': request_data_source {
type: REQUEST_SOURCE
request_data_options {
schema {
name: "val_to_add"
value_type: INT64
}
schema {
name: "val_to_add_2"
value_type: INT64
}
}
name: "vals_to_add"
}
} -> {'driver_hourly_stats_fresh': feature_view_projection {
feature_view_name: "driver_hourly_stats_fresh"
feature_columns {
name: "conv_rate"
value_type: FLOAT
}
feature_columns {
name: "avg_daily_trips"
value_type: INT64
}
timestamp_field: "created"
created_timestamp_column: "created"
batch_source {
type: BATCH_FILE
timestamp_field: "event_timestamp"
created_timestamp_column: "created"
file_options {
uri: "data/driver_stats.parquet"
}
name: "driver_hourly_stats_source"
}
}
, 'vals_to_add': request_data_source {
type: REQUEST_SOURCE
request_data_options {
schema {
name: "val_to_add"
value_type: INT64
}
schema {
name: "val_to_add_2"
value_type: INT64
}
}
name: "vals_to_add"
}
}
Updated feature service driver_activity_v3
features: [feature_view_name: "driver_hourly_stats_fresh"
feature_columns {
name: "conv_rate"
value_type: FLOAT
}
feature_columns {
name: "acc_rate"
value_type: FLOAT
}
feature_columns {
name: "avg_daily_trips"
value_type: INT64
}
timestamp_field: "created"
created_timestamp_column: "created"
batch_source {
type: BATCH_FILE
timestamp_field: "event_timestamp"
created_timestamp_column: "created"
file_options {
uri: "data/driver_stats.parquet"
}
name: "driver_hourly_stats_source"
}
, feature_view_name: "transformed_conv_rate_fresh"
feature_columns {
name: "conv_rate_plus_val1"
value_type: DOUBLE
}
feature_columns {
name: "conv_rate_plus_val2"
value_type: DOUBLE
}
] -> [feature_view_name: "driver_hourly_stats_fresh"
feature_columns {
name: "conv_rate"
value_type: FLOAT
}
feature_columns {
name: "avg_daily_trips"
value_type: INT64
}
timestamp_field: "created"
created_timestamp_column: "created"
batch_source {
type: BATCH_FILE
timestamp_field: "event_timestamp"
created_timestamp_column: "created"
file_options {
uri: "data/driver_stats.parquet"
}
name: "driver_hourly_stats_source"
}
, feature_view_name: "transformed_conv_rate_fresh"
feature_columns {
name: "conv_rate_plus_val1"
value_type: DOUBLE
}
feature_columns {
name: "conv_rate_plus_val2"
value_type: DOUBLE
}
]
No changes to infrastructure
```
You can imagine how bad this gets with larger changes.
**Describe the solution you'd like**
Either make the text based plan more concise and similar to Terraform by hiding unnecessary information such as values that do not change.
At the very least we could have JSON output with `"from"` and `"to"` fields for each object. This would be equally verbose but at least machine parsable which the existing output is not either.
**Describe alternatives you've considered**
* Terraform style plan that only outputs changed fields in each object/resource.
* HTML rendered plan which can be emailed to project owners/posted to PR requests by some CI process (or otherwise displayed in a browser).
**Additional context**
N/a
Thanks!
Contributor guide
Assessment
This issue has not been assessed yet.