Custom scalars dashboard tries to load data for irrelevant runs

The tf-line-chart-data-loader elements used in the scalars plugin via tf-scalar-card take a runs list and tag that are specific to that chart, i.e. the tag is the one the chart is displaying and the runs are those that the backend computed have data for this tag, and which are served under the /tags endpoint as a JSON object mapping runs to tags to tag metadata:
https://github.com/tensorflow/tensorboard/blob/1.5.0/tensorboard/plugins/scalar/tf_scalar_dashboard/tf-scalar-dashboard.html#L141

This ensures that when the chart loads its data, it issues a /scalars request only for run+tag combinations that we expect to actually produce data.

However, the custom scalars plugin always passes the entire list of known runs (i.e. everything the run selector has selected) to its tf-line-chart-data-loader-containing elements:
https://github.com/tensorflow/tensorboard/blob/1.5.0/tensorboard/plugins/custom_scalar/tf_custom_scalar_dashboard/tf-custom-scalar-dashboard.html#L174,L189

This results in one /scalars request per selected run for each chart on the custom scalars dashboard, regardless of whether there is any data to serve for that tag-regex and run combination.

In a case where there are 100 runs (all of which are selected by default) and only 3 of them are being used for the custom scalars dashboard, this means we send 100 requests instead of 3. Over a slow network connection this is quite noticeably slow, and causes the custom scalar chart to show the "loader" circle gif for much longer than would otherwise seem needed. One easy way to repro this behavior is just to run the scalars demo code and the custom scalars demo code, merge the two logdirs, and then load the result (since the scalars demo code creates 18 runs).

I think a more efficient approach would be to have the custom scalars backend basically load from the scalars PluginRunToTagToContent an inverted mapping of tag -> run, then match each custom chart spec's tag regexes/etc. against the tags in that list to get a list of applicable runs, and expose this information to the dashboard in addition to /layout. Then each card would get its own set of applicable runs, and intersect them with the runs selected in the sidebar to produce the runs that should be loaded. Note that the set of runs per chart would need to be updated on refresh like for the scalars dashboard since it might grow over time.

Start with tf-custom-scalar-dashboard.html and tf-scalar-dashboard.html, then trace how tf-line-chart-data-loader receives runs and how the /tags, /scalars, and /layout endpoints are used. Done means each custom-scalar chart requests data only for selected runs applicable to its tag patterns, with those run sets refreshed as data grows.