googleapis / googleapis/google-cloud-java

[java-bigquerystorage] JsonToProtoMessage accepts byte[] for repeated BYTES but not scalar BYTES

Đang mở
#13,979 0 bình luận 0 reaction 0 người được giao Xem trên GitHub
Ngôn ngữ chính
Java
Star
2.1k
Fork
1.2k
Merge trung bình
1 ngày 23 giờ
Pull request đã merge (30 ngày)
154

Mô tả

## Issue Details

For a `BYTES` column, `JsonToProtoMessage` accepts a `byte[]` value in the repeated path but not in
the scalar path:

- `fillRepeatedField`'s `case BYTES:` accepts `ByteString`, `byte[]` and `JSONArray`.
- `fillField`'s `case BYTES:` accepts `ByteString` and `JSONArray` only — a `byte[]` falls through
to the "wrong field type" error.

Nothing in the API suggests a scalar column should be more restrictive than a repeated one of the
same type, so this reads as an oversight rather than an intentional restriction.

I have a pull request ready for this, combined with the base64 fix in the same branch (see below).

## Environment

- OS Type and Version: macOS 26.5.2
- Java Version and JDK Vendor: OpenJDK 17.0.19, Eclipse Temurin (17.0.19+10)
- (If using GraalVM) GraalVM Version: n/a

Not deployed on GCP — this reproduces entirely locally, since the failure is in client-side
conversion before any RPC is made.

## Dependencies

- Libraries-Bom: `com.google.cloud:libraries-bom:26.85.1`
- Client library: `com.google.cloud:google-cloud-bigquerystorage:3.30.0` (resolved by the BOM
above)
- `com.google.protobuf:protobuf-java:4.33.2`
- `org.json:json` (the `JSONObject` parameter type of `convertToProtoMessage`)

No RPC is made, so gax and auth are not involved in the failure.

## Reproducer

Single file, no GCP credentials or resources needed. Run with `google-cloud-bigquerystorage` on the
classpath.

```java
import com.google.cloud.bigquery.storage.v1.*;
import com.google.protobuf.ByteString;
import com.google.protobuf.Descriptors;
import org.json.JSONArray;
import org.json.JSONObject;

public class BytesArrProbe {
static TableFieldSchema f(String n, TableFieldSchema.Type t, TableFieldSchema.Mode m) {
return TableFieldSchema.newBuilder().setName(n).setType(t).setMode(m).build();
}

public static void main(String[] args) throws Exception {
TableSchema schema =
TableSchema.newBuilder()
.addFields(f("blob", TableFieldSchema.Type.BYTES, TableFieldSchema.Mode.NULLABLE))
.addFields(f("blobs", TableFieldSchema.Type.BYTES, TableFieldSchema.Mode.REPEATED))
.build();
Descriptors.Descriptor d =
BQTableSchemaToProtoDescriptor.convertBQTableSchemaToProtoDescriptor(schema);

byte[] hi = new byte[] {104, 105};

probe(d, schema, "scalar byte[]", new JSONObject().put("blob", hi));
probe(d, schema, "scalar ByteString", new JSONObject().put("blob", ByteString.copyFrom(hi)));
probe(d, schema, "repeated byte[]", new JSONObject().put("blobs", new JSONArray().put(hi)));
probe(
d,
schema,
"repeated ByteString",
new JSONObject().put("blobs", new JSONArray().put(ByteString.copyFrom(hi))));
}

static void probe(Descriptors.Descriptor d, TableSchema s, String label, JSONObject json) {
try {
Object msg = JsonToProtoMessage.INSTANCE.convertToProtoMessage(d, s, json, false);
System.out.println("OK " + label + " :: " + msg.toString().replace("\n", " ").trim());
} catch (Exception e) {
System.out.println(
"FAIL " + label + " :: " + e.getClass().getSimpleName() + ": " + e.getMessage());
}
}
}
```

Steps:

1. Put `google-cloud-bigquerystorage:3.30.0` (and its transitive dependencies) on the classpath.
2. Run the class above.
3. Observe that the scalar `byte[]` case fails while the repeated `byte[]` case succeeds.

## Logs and Stack Trace

Output of the reproducer, verbatim:

```
FAIL scalar byte[] :: RowIndexToErrorException: The map of row index to error message is {0=Field root.blob failed to convert to BYTES. Error: JSONObject does not have a bytes field at root.blob.}
OK scalar ByteString :: blob: "hi"
OK repeated byte[] :: blobs: "hi"
OK repeated ByteString :: blobs: "hi"
```

## Behavior

- **When did the issue begin?** Not a regression; the two branches have differed for a long time.
- **Is it flaky?** No, fully deterministic and reproducible offline.
- **Related to volume of data?** No.

## Note

This sits in the same `case BYTES:` branch as the base64 problem I am filing alongside it, so the
pull request fixes both. Happy to split it into a separate pull request if you would rather review
them independently.

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Hướng nghiên cứu

Start at JsonToProtoMessage's fillField BYTES branch and compare it with fillRepeatedField's BYTES branch. Use the provided BytesArrProbe to reproduce the scalar byte[] failure locally without credentials. Done means scalar byte[] input converts successfully while the existing ByteString and repeated cases continue to work.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Đánh giá

Công nghệ
google-cloud, java
Lĩnh vực
api
Loại issue
Lỗi
Độ khó
1/5
Thời gian dự kiến
1-3 giờ
Mức độ hoạt động
Đình trệ
Độ rõ ràng
Đặc tả rõ ràng
Mức phù hợp với người mới
35/100

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.