roachperf/gce: data collection needs to use GCP project specific bucket

Part of #77644.

## Motivation
As a consequence of the GCP project migration, roachperf will need to have multiple projects' GCS buckets as source and needs a way of constructing views on those sources that is not destructive.
* Sources: `gs://cockroach-nightly[-]`, `gs://cockroach-nightly[-]-crl-e2e-infra-staging`, `gs://cockroach-nightly[-]-crl-e2e-infra`
* possible Views:
* `gs://cockroach-nightly[-]`
* `gs://cockroach-nightly[-]` + `gs://cockroach-nightly[-]-crl-e2e-infra-staging`
* `gs://cockroach-nightly[-]` + `gs://cockroach-nightly[-]-crl-e2e-infra`
* `gs://cockroach-nightly[-]-crl-e2e-infra`
* `gs://cockroach-nightly[-]-crl-e2e-infra-staging`

Note: `gs://cockroach-nightly[-]` is shorthand for `gs://cockroach-nightly`, `gs://cockroach-nightly-aws`, `gs://cockroach-nightly-azure`, `gs://cockroach-nightly-ibm` for `gce`, `aws`, `azure`, `ibm` artifacts respectively. i.e. GCE is implied if there's not cloud provider postfix on the bucket name

Currently, this there is no way to ingest data from more than one bucket and separating generated artifacts. Also artifact organization on the file system could be improved. Currently roachperf pulls data from `gs://cockroach-nightly` which is in the `cockroach-ephemeral` GCP project. This project is being deprecated and new buckets will be provisioned in the new staging and prod GCP projects respectively.
* Alternatively, we could have a roachperf k8s cluster per GCP project i.e. `crl-e2e-infra-staging`, and `crl-e2e-infra` could be served by their own roachperf instance. The main downside to this is that setting up and downloading artifacts to a new cluster and generating new artifacts would take days. The current goal is to get roachperf working with multiple GCS sources and views as soon as possible.

## Current `/data` layout

No source bucket implied from directory structure which made sense because there was only `gs://cockroach-nightly[-]`. `master` is currently implicit: its files live directly under `/data`. Each architecture or release is a separate sibling directory.

```text
/data/
├── gce/ # implicit master, default architecture
├── aws/
├── azure/
├── ibm/
├── stability/
├── detail/
├── perf.js
├── perf.js.gz
│
├── arm64/ # implicit master, ARM64
│ ├── gce/
│ ├── aws/
│ ├── azure/
│ ├── ibm/
│ ├── stability/
│ ├── detail/
│ └── perf.js
│
├── fips/ # implicit master, FIPS
│ └── ...
│
├── s390x/ # implicit master, S390x
│ ├── gce/
│ ├── aws/
│ ├── azure/
│ ├── ibm/ # currently the meaningful S390x data
│ ├── stability/
│ ├── detail/
│ └── perf.js
│
├── release-26.2/
│ ├── gce/
│ ├── aws/
│ ├── azure/
│ ├── ibm/
│ ├── stability/
│ ├── detail/
│ └── perf.js
│
├── release-26.2-arm64/
├── release-26.2-fips/
├── release-26.1/
├── release-25.4/
└── ...
```

## Proposed layout

Use a logical source/environment as the first dimension, then an explicit branch/architecture target. A source contains one bucket URI per cloud; it is not itself a literal GCS bucket because there is an individual bucket per cloud e.g. `gs://cockroach-nightly-crl-e2e-infra-staging[-]`.

Also it doesn't make sense that every other release branch besides `master` gets its own subdirectory, so `master[-arm64,-fips,-s390x]` will be added as subdirectories.

Each source group is identified by its GCE bucket name. This uses the actual resource name for both legacy (`cockroach-nightly`) and the new projects; the provider-specific bucket names are explicit in the source configuration.

```text
/data/
├── sources/ # downloaded raw artifacts
│ ├── cockroach-nightly/ # GCS Bucket* (technically represents 4 buckets, 1 for each cloud provider)
│ │ ├── master/
│ │ │ ├── gce/artifacts/...
│ │ │ ├── aws/artifacts/...
│ │ │ ├── azure/artifacts/...
│ │ │ └── ibm/artifacts/...
│ │ ├── master-arm64/
│ │ ├── master-fips/
│ │ ├── master-s390x/
│ │ ├── release-26.2/
│ │ └── ...
│ │
│ ├── cockroach-nightly-crl-e2e-infra-staging/
│ │ ├── master/
│ │ │ ├── gce/artifacts/...
│ │ │ ├── aws/artifacts/...
│ │ │ ├── azure/artifacts/...
│ │ │ └── ibm/artifacts/...
│ │ ├── master-arm64/
│ │ ├── master-fips/
│ │ ├── master-s390x/
│ │ ├── release-26.2/
│ │ └── ...
│ │
│ └── cockroach-nightly-crl-e2e-infra/
│ └── ...
│
└── views/ # generated, HTTP-served assets
├── cockroach-nightly/
│ ├── master/
│ │ ├── perf.js
│ │ ├── perf.js.gz
│ │ └── detail/
│ │ ├── gce/...
│ │ ├── aws/...
│ │ ├── azure/...
│ │ └── ibm/...
│ ├── master-arm64/
│ ├── master-fips/
│ ├── master-s390x/
│ ├── release-26.2/
│ └── ...
│
├── cockroach-nightly-crl-e2e-infra-staging/
│ ├── master/
│ │ ├── perf.js
│ │ ├── perf.js.gz
│ │ └── detail/
│ │ └── ...
│ └── ...
│
├── cockroach-nightly-crl-e2e-infra/
│ └── ...
│
├── legacy-staging/
│ ├── master/
│ │ ├── perf.js
│ │ └── perf.js.gz
│ ├── master-arm64/
│ ├── master-fips/
│ ├── master-s390x/
│ ├── release-26.2/
│ └── ...
│
└── legacy-prod/
├── master/
│ ├── perf.js
│ └── perf.js.gz
├── master-arm64/
├── master-fips/
├── master-s390x/
├── release-26.2/
└── ...
```
`legacy-staging` and `legacy-prod` are concrete combined-view names defined in the configuration below.

For a combined view, each run's `Path` in `perf.js` is a source-qualified,
same-origin HTTP path. For example:

```text
/views/cockroach-nightly/master/detail/gce//data.json
/views/cockroach-nightly-crl-e2e-infra-staging/master/detail/gce//data.json
```

Therefore each combined-view directory (for example,
`views/legacy-staging//`) intentionally contains only `perf.js` and
`perf.js.gz`; it reuses the detail files already generated and served by each
source view.

## Implementation
Roachperf currently scans 4 buckets `gs://cockroach-nightly[-]` and downloads test run data to create metric data served by roachperf. The multi-source implementation PR will be stacked on [roachperf#232](https://github.com/cockroachdb/roachperf/pull/232), which removes the separate AWS S3 stability-data source; stability is therefore not part of this proposed layout.

Each source group's GCE bucket name is its source ID. Its data is placed in `/data/sources//` and its generated artifacts are placed in `/data/views//`. This new layout disambiguates benchmark data from roachperf-generated artifacts.

For a configured target, roachperf derives the remote artifact prefix from the source's `artifacts_prefix`: for example, `master` uses `artifacts`, `master-arm64` uses `artifacts-arm64`, and `release-26.2` uses `artifacts-release-26.2`.

Downloading from a non `gs://cockroach-nightly` will be defined in a `.yaml` file.
* a CLI argument could also be used during runtime, but a configuration file makes things upfront and transparent

Each configured source/target must be synchronized once per sync cycle. Source
and combined views must reuse that local source data rather than independently
listing or fetching the same GCS prefix.

Generating the roachperf generated test-specific artifacts remains the same; each `/data/views//` directory can have its own `perf.js` and `perf.js.gz`.

If a combined view is desired i.e. using data from both `gs://cockroach-nightly[-]` + `gs://cockroach-nightly[-]-crl-e2e-infra-staging`, this will create a combined `perf.js` (and `perf.js.gz`) which will be placed in `/data/views///`.
* How would creating these artifacts take?

For serving this combined `perf.js` (and `perf.js.gz`), a config file can be used, example below
* a CLI argument could also be used during runtime, but a configuration file makes things upfront and transparent

### Static view routing

Each configured `/views///` URL must serve the standard roachperf HTML, CSS, and JavaScript bundle in addition to that view's `perf.js`, `perf.js.gz`, and detail files. The implementation may copy or symlink the static bundle into each view directory or use NGINX rewrites or aliases, but relative asset URLs (for example `css/bootstrap.min.css`, `js/d3.min.js`, and `perf-app.js`) must resolve correctly for source and combined views.

The following invariant guarantees we will not have any collision in the combined view with the legacy bucket and a new bucket. We should never have a date collision because cockroach-ephemeral's cockroach-nightly will only contain historical data from pre #77644 implementation
```
Merge invariant: for a given target, cloud, normalized test name, date, and
warehouse count, at most one configured source may contain a run. The combined
view must fail generation with a clear conflict error if this invariant is
violated; it must not silently select a source based on processing order.
```

The motivation behind the above approach is that if we simply did a many GCS buckets to single `/data` directory approach, we would be combining benchmark results from both `crl-e2e-infra-staging` and `crl-e2e-infra` which doesn't make sense. This way we keep the data separate, generate each configured view during the sync cycle, and serve the resulting pre-generated static view.

The `sync` k8s job will need to download each configured source into `/data/sources///`, including `cockroach-nightly-crl-e2e-infra-staging` and, when provisioned, `cockroach-nightly-crl-e2e-infra`.

### Proposed config file

```yaml
# /etc/roachperf/views.yaml

storage:
sources_root: /data/sources
views_root: /data/views
public_views_prefix: /views

sources:
cockroach-nightly:
clouds:
gce:
bucket: gs://cockroach-nightly
artifacts_prefix: artifacts
aws:
bucket: gs://cockroach-nightly-aws
artifacts_prefix: artifacts
azure:
bucket: gs://cockroach-nightly-azure
artifacts_prefix: artifacts
ibm:
bucket: gs://cockroach-nightly-ibm
artifacts_prefix: artifacts

cockroach-nightly-crl-e2e-infra-staging:
clouds:
gce:
bucket: gs://cockroach-nightly-crl-e2e-infra-staging
artifacts_prefix: artifacts
aws:
bucket: gs://cockroach-nightly-aws-crl-e2e-infra-staging
artifacts_prefix: artifacts
azure:
bucket: gs://cockroach-nightly-azure-crl-e2e-infra-staging
artifacts_prefix: artifacts
ibm:
bucket: gs://cockroach-nightly-ibm-crl-e2e-infra-staging
artifacts_prefix: artifacts

# Enable after these production buckets are provisioned and readable.
# cockroach-nightly-crl-e2e-infra:
# clouds:
# gce:
# bucket: gs://cockroach-nightly-crl-e2e-infra
# artifacts_prefix: artifacts
# aws:
# bucket: gs://cockroach-nightly-aws-crl-e2e-infra
# artifacts_prefix: artifacts
# azure:
# bucket: gs://cockroach-nightly-azure-crl-e2e-infra
# artifacts_prefix: artifacts
# ibm:
# bucket: gs://cockroach-nightly-ibm-crl-e2e-infra
# artifacts_prefix: artifacts

views:
cockroach-nightly:
sources: [cockroach-nightly]

cockroach-nightly-crl-e2e-infra-staging:
sources: [cockroach-nightly-crl-e2e-infra-staging]

legacy-staging:
sources: [cockroach-nightly, cockroach-nightly-crl-e2e-infra-staging]

# legacy-prod:
# sources: [cockroach-nightly, cockroach-nightly-crl-e2e-infra]

targets:
master: {}
master-arm64: { architecture: arm64 }
master-fips: { architecture: fips }
master-s390x: { architecture: s390x }
release-26.2: { release: "26.2" }
```
## Infra

Currently, a SA `roachperf-sync` (in `cockroach-ephemeral`) is used to access `gs://cockroach-nightly`(?). We most likely need new SA's in the respective `crl-e2e-infra-staging` and `crl-e2e-infra` projects to read from `gs://cockroach-nightly-crl-e2e-infra-staging` and `gs://cockroach-nightly-crl-e2e-infra` respectively.
* https://console.cloud.google.com/iam-admin/serviceaccounts/details/101702297168260265954?project=cockroach-ephemeral
* The secret to access the service account can be accessed via `kubectl get secret roachperf-sync -o json`, the secret is the value of `gcloud.json` key. (verify this)

## Other Changes
In addition to roachperf, there are references to the GCS bucket in other locations that may need to be addressed.
```
FYI:

## Readers that must be addressed

Yes, there are several readers beyond the nightly scripts:

- pkg/cmd/roachprod/cli/update.go:21 hard-codes the bucket used by roachprod update.
- scripts/roachprod-get-latest.sh:21 does the same.
- The compile cache reads build-cache/ before writing it.
- roachperf (https://github.com/cockroachdb/roachperf/blob/88f0d167ff64b5b24aae0399aada9cabfb5c9e65/main.go) hard-codes gs://cockroach-nightly/artifacts.
- The data-analytics pipelines hard-code both cockroach-ephemeral and cockroach-nightly, including raw_direct_gce.yaml (https://github.com/cockroachlabs/data-analytics/blob/fadb56dd2049b1421293beb17e84836ecabb266e/data-pipelines/resources/roachperf/raw_direct_gce.yaml), raw_histograms_gce.yaml, and raw_summary_stats_gce.yaml.
- cockroachlabs/RoachperfStats has additional GCE ingestion configurations that should be checked with that owner.
- There are a few archived/manual references in support and performance-team repositories. Those are not cutover blockers but should be notified or updated if still used.

The existing bucket IAM confirms active external consumers: roachperf-etl, data-pipeline identities, om-converter, and other service accounts have access.
```

Verification
* Need to test e2e upload from TeamCity runner to `cockroach-nightly-` and then roachperf pulling down the test data, and see roachperf serving the desired combined view.

This [PR](https://github.com/cockroachlabs/cockroach/pull/3180/changes#diff-111c830ea80999136efa4cf70faf2140c986c7a4de2f7641d093d7e95c85fdfbR14) changes the CI build scripts to use the new GCS bucket names
This [PR](https://github.com/cockroachlabs/cockroach/pull/3212) adds a branch name override for benchmark upload to GCS in CI scripts after roachtest invocation.

Notes:
* During filesystem modifications, the `sync` k8s job must not be running.