googleapis / googleapis/google-cloud-java
[java-bigquerystorage] JsonToProtoMessage accepts byte[] for repeated BYTES but not scalar BYTES
- Lingua principale
- Java
- Stelle
- 2.1k
- Fork
- 1.2k
- Merge medio
- 1g 23h
- PR unite (30g)
- 154
Descrizione
## Issue Details
For a `BYTES` column, `JsonToProtoMessage` accepts a `byte[]` value in the repeated path but not in
the scalar path:
- `fillRepeatedField`'s `case BYTES:` accepts `ByteString`, `byte[]` and `JSONArray`.
- `fillField`'s `case BYTES:` accepts `ByteString` and `JSONArray` only — a `byte[]` falls through
to the "wrong field type" error.
Nothing in the API suggests a scalar column should be more restrictive than a repeated one of the
same type, so this reads as an oversight rather than an intentional restriction.
I have a pull request ready for this, combined with the base64 fix in the same branch (see below).
## Environment
- OS Type and Version: macOS 26.5.2
- Java Version and JDK Vendor: OpenJDK 17.0.19, Eclipse Temurin (17.0.19+10)
- (If using GraalVM) GraalVM Version: n/a
Not deployed on GCP — this reproduces entirely locally, since the failure is in client-side
conversion before any RPC is made.
## Dependencies
- Libraries-Bom: `com.google.cloud:libraries-bom:26.85.1`
- Client library: `com.google.cloud:google-cloud-bigquerystorage:3.30.0` (resolved by the BOM
above)
- `com.google.protobuf:protobuf-java:4.33.2`
- `org.json:json` (the `JSONObject` parameter type of `convertToProtoMessage`)
No RPC is made, so gax and auth are not involved in the failure.
## Reproducer
Single file, no GCP credentials or resources needed. Run with `google-cloud-bigquerystorage` on the
classpath.
```java
import com.google.cloud.bigquery.storage.v1.*;
import com.google.protobuf.ByteString;
import com.google.protobuf.Descriptors;
import org.json.JSONArray;
import org.json.JSONObject;
public class BytesArrProbe {
static TableFieldSchema f(String n, TableFieldSchema.Type t, TableFieldSchema.Mode m) {
return TableFieldSchema.newBuilder().setName(n).setType(t).setMode(m).build();
}
public static void main(String[] args) throws Exception {
TableSchema schema =
TableSchema.newBuilder()
.addFields(f("blob", TableFieldSchema.Type.BYTES, TableFieldSchema.Mode.NULLABLE))
.addFields(f("blobs", TableFieldSchema.Type.BYTES, TableFieldSchema.Mode.REPEATED))
.build();
Descriptors.Descriptor d =
BQTableSchemaToProtoDescriptor.convertBQTableSchemaToProtoDescriptor(schema);
byte[] hi = new byte[] {104, 105};
probe(d, schema, "scalar byte[]", new JSONObject().put("blob", hi));
probe(d, schema, "scalar ByteString", new JSONObject().put("blob", ByteString.copyFrom(hi)));
probe(d, schema, "repeated byte[]", new JSONObject().put("blobs", new JSONArray().put(hi)));
probe(
d,
schema,
"repeated ByteString",
new JSONObject().put("blobs", new JSONArray().put(ByteString.copyFrom(hi))));
}
static void probe(Descriptors.Descriptor d, TableSchema s, String label, JSONObject json) {
try {
Object msg = JsonToProtoMessage.INSTANCE.convertToProtoMessage(d, s, json, false);
System.out.println("OK " + label + " :: " + msg.toString().replace("\n", " ").trim());
} catch (Exception e) {
System.out.println(
"FAIL " + label + " :: " + e.getClass().getSimpleName() + ": " + e.getMessage());
}
}
}
```
Steps:
1. Put `google-cloud-bigquerystorage:3.30.0` (and its transitive dependencies) on the classpath.
2. Run the class above.
3. Observe that the scalar `byte[]` case fails while the repeated `byte[]` case succeeds.
## Logs and Stack Trace
Output of the reproducer, verbatim:
```
FAIL scalar byte[] :: RowIndexToErrorException: The map of row index to error message is {0=Field root.blob failed to convert to BYTES. Error: JSONObject does not have a bytes field at root.blob.}
OK scalar ByteString :: blob: "hi"
OK repeated byte[] :: blobs: "hi"
OK repeated ByteString :: blobs: "hi"
```
## Behavior
- **When did the issue begin?** Not a regression; the two branches have differed for a long time.
- **Is it flaky?** No, fully deterministic and reproducible offline.
- **Related to volume of data?** No.
## Note
This sits in the same `case BYTES:` branch as the base64 problem I am filing alongside it, so the
pull request fixes both. Happy to split it into a separate pull request if you would rather review
them independently.
Guida per i contributori
Apri la guida per i contributori
Direzione di ricerca
Start at JsonToProtoMessage's fillField BYTES branch and compare it with fillRepeatedField's BYTES branch. Use the provided BytesArrProbe to reproduce the scalar byte[] failure locally without credentials. Done means scalar byte[] input converts successfully while the existing ByteString and repeated cases continue to work.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Valutazione
- Stack tecnologico
- google-cloud, java
- Ambito
- api
- Tipo di issue
- Bug
- Difficoltà
- 1/5
- Tempo stimato
- 1-3 ore
- Stato di attività
- Ferma
- Chiarezza
- Specificata chiaramente
- Idoneità per principianti
- 35/100