googleapis / googleapis/google-cloud-python

google-cloud-bigquery: support column-level data policies (dataPolicies) in SchemaField

Đang mở
#17,617 0 bình luận 0 reaction 0 người được giao Xem trên GitHub
Ngôn ngữ chính
Python
Star
5.4k
Fork
1.8k
Merge trung bình
3 ngày 4 giờ
Pull request đã merge (30 ngày)
122

Mô tả

## Feature request

`SchemaField` supports `policy_tags` (Data Catalog column-level security) but has no equivalent for **column-level data policies** (`dataPolicies`), the v2 data-masking / raw-data-access policies bound directly to a column. This makes it impossible to read, set, or modify a column's data policies through the ergonomic client — callers must drop down to the raw REST API (`tables.get` / `tables.patch`) plus DDL.

## Background

The BigQuery REST API exposes `dataPolicies` on `TableFieldSchema` — `IList`, *"Optional. Data policies attached to this field, used for field-level access control"* (see the [`tables` resource reference](https://cloud.google.com/bigquery/docs/reference/rest/v2/tables)). This is distinct from `policyTags`. The separate `bigquery-datapolicies` client manages the *policy resources*, but not the *binding* of a policy to a column, which is a table-schema operation.

## What's missing in `SchemaField`

In `packages/google-cloud-bigquery/google/cloud/bigquery/schema.py`:

- `__init__` accepts `policy_tags` but has no `data_policies` parameter.
- There is a `policy_tags` property getter but no `data_policies` getter.
- `_key()` does not account for data policies.

`from_api_repr` stores the whole API dict in `_properties`, so a pure read round-trip retains `dataPolicies` as an opaque key — but there is no supported way to *read* it (no getter) or to *set/modify* it (no constructor param / setter) without reaching into the private `_properties`.

## Repro

```python
from google.cloud import bigquery # 3.38.0

f = bigquery.SchemaField("ssn", "STRING")
# No way to attach a data policy:
# bigquery.SchemaField("ssn", "STRING", data_policies=[...]) # unsupported
# f.data_policies # AttributeError

# Reading an existing field:
table = client.get_table("proj.ds.tbl") # column has a data policy bound out-of-band
field = table.schema[0]
# field.data_policies -> no such attribute; only field._properties.get("dataPolicies") (private)
```

## Requested

Add `data_policies` to `SchemaField` mirroring `policy_tags`:
- constructor param + property getter,
- serialized into `_properties["dataPolicies"]`,
- included in `_key()` for equality/hashing,
- unit coverage in `packages/google-cloud-bigquery/tests/unit/test_schema.py`.

## Related gotcha (worth documenting either way)

When manipulating `dataPolicies` via `tables.patch`/`tables.update`, an **empty** `dataPolicies` array is silently ignored (treated as "no change"), so a column's **last** data policy cannot be removed through the schema API — only DDL (`ALTER TABLE ... ALTER COLUMN SET OPTIONS (data_policies=[])`) clears it. Non-empty updates (add / reduce to ≥1) do work via patch.

## Environment

- `google-cloud-bigquery==3.38.0` (also confirmed against the latest reference docs, which list the same `SchemaField` params — no `data_policies`).

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Hướng nghiên cứu

Start in packages/google-cloud-bigquery/google/cloud/bigquery/schema.py by tracing how SchemaField handles policy_tags, then inspect the related unit coverage in packages/google-cloud-bigquery/tests/unit/test_schema.py. Add coverage for constructing, reading, serializing, and comparing data_policies, and run test_schema.py to verify the requested behavior.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Đánh giá

Công nghệ
python
Lĩnh vực
api, databases
Loại issue
Tính năng
Độ khó
3/5
Thời gian dự kiến
1-2 ngày
Mức độ hoạt động
Ít trao đổi
Độ rõ ràng
Đặc tả rõ ràng
Mức phù hợp với người mới
82/100

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.