dbt-labs / dbt-labs/dbt-cloud-openapi-spec
API v3: Credentials endpoint requires undocumented credential_details payload for BigQuery
- Dominant language
- Python
- Stars
- 12
- Forks
- 10
- PR merge metrics
- No merged PRs in 30d
Description
## Summary
The v3 Admin API `Create Credentials` endpoint for BigQuery adapter credentials requires a nested `credential_details` payload that is **not documented** in the current OpenAPI spec or examples.
As a result, customers who follow the published schema cannot create working BigQuery v1 deployment credentials via API without reverse‑engineering browser calls.
---
## Affected endpoint
- `POST /api/v3/accounts/{account_id}/projects/{project_id}/credentials/`
- `PATCH /api/v3/accounts/{account_id}/projects/{project_id}/credentials/{id}/`
When creating **BigQuery v1 deployment credentials** with:
- `type: "adapter"`
- BigQuery v1 adapter
the API expects a `credential_details` object with:
- `adapter_version: "bigquery_v1"`
- `fields.*.metadata` / `fields.*.value` entries for:
- `auth_type`
- `schema`, `target_name`, `threads`
- BigQuery service account JSON fields (`project_id`, `private_key_id`, `private_key`, `client_email`, `client_id`, `auth_uri`, `token_uri`, `auth_provider_x509_cert_url`, `client_x509_cert_url`, etc.)
This structure is generated by the dbt Cloud UI but is not described anywhere in the OpenAPI spec.
---
## Current behavior
- The v3 OpenAPI spec for the Credentials endpoints does **not** describe `credential_details` or its nested fields for adapter credentials.
- If a customer follows the documented schema and omits `credential_details`, they can:
- End up with credentials objects that look valid in the UI but fail at runtime with:
- `Failed to generate profile due to incorrect credentials` (e.g. when `private_key.value` is empty).
- If a customer guesses and sends `credential_details` to the wrong endpoint or with the wrong type, they get:
- 400s like:
`Additional properties are not allowed ('credential_details' was unexpected)`
- The only reliable way to discover the required payload today is to:
1. Create credentials via the UI.
2. Inspect the network calls / `GET /credentials/{id}/` response.
3. Copy that structure into API scripts.
---
## Expected behavior
- The v3 OpenAPI spec and generated docs should:
- Clearly document that **adapter credentials** for warehouses like BigQuery v1 use `type: "adapter"` plus a `credential_details` object.
- Include the expected schema for `credential_details` (at least for BigQuery v1):
- `adapter_version: "bigquery_v1"`
- `fields..metadata`
- `fields..value`
- Provide at least one **example** payload for:
- BigQuery v1 deployment credentials using service account JSON.
This would allow customers to build automation purely from the published spec, without having to reverse‑engineer UI network calls.
---
## Customer impact
- Customer: **Agile Lab** ([Agile Lab ticket in Zendesk](https://dbtcloud.zendesk.com/agent/tickets/116538)).
- Goal: Use the v3 Admin API to automate project setup:
- One‑time creation of a BigQuery connection via the UI (uploading the service account JSON).
- Then programmatically create **deployment credentials and environments** via API only.
- Problems they hit:
- The OpenAPI spec does not mention `credential_details`, so they initially followed the documented schema and sent a simple body.
- In practice, the API requires a nested `credential_details.fields.*` payload (including `private_key`), which they had to infer from browser network calls.
- Incorrect or incomplete payloads produced:
- 400 responses: `Additional properties are not allowed ('credential_details' was unexpected)` on some calls.
- Runtime failures: `Failed to generate profile due to incorrect credentials` when `private_key.value` was empty.
- Credentials created via API appeared valid in the UI, but job runs only succeeded after manually editing & saving credentials in the UI.
Net effect: extra debugging effort, brittle automation, and confusion because the **effective API contract does not match the published spec or examples**.
---
## Requested changes
1. **Update `openapi-v3.yaml`** for the Credentials endpoints to:
- Add the `credential_details` object to the request body schema for adapter credentials.
- Document the nested structure for `fields.*.metadata` and `fields.*.value` for BigQuery v1 (and, ideally, other adapters that use this pattern)
2. **Add example payloads** to the v3 docs for:
- Creating BigQuery v1 deployment credentials via the Admin API, using service account JSON.
- Optionally, a short description in the docs page clarifying the intended flow:
- Connection via UI (or API),
- Adapter credentials with `credential_details`,
- Environment referencing `credentials_id`
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.