googleapis / googleapis/google-cloud-java

[java-bigquery] BigQuery: data is sometimes not inserted in tables

未关闭
#12,143 2 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
api: bigquery priority: p3 type: bug
主要语言
Java
星标
2.1k
派生
1.2k
平均合并
1 天 23 小时
30 天内合并 PR
157

描述

Hi,

I'm having random data missing/not showing up in BigQuery issues. I would seemingly write data without issues, but SELECT * would return nothing.

1. Have tried latest version but still same issue.

```

com.google.cloud
google-cloud-bigquery
2.28.0

```

2. Linux and Windows
3. JDK 8.202

#### Steps to reproduce

Issue is random, can't reproduce reliably. The writes are batched to 1-2k rows and 99% they work.

The control panel shows no errors. None of the quotas show up as exceeded. I get no exceptions and response.getInsertErrors() is always null. The issue seems to happen when doing intensive writes, as less batches per minute of the same size work fine.

Any ideas what's wrong? thank you.

#### Code example

```java
GoogleCredential credential = new GoogleCredential.Builder().setTransport(new NetHttpTransport())
.setJsonFactory(new JacksonFactory())
.setClientSecrets("clientId", "clientSecret")
.build()
.setFromTokenResponse(new TokenResponse().setRefreshToken("refreshToken");

Bigquery bigquery = new Bigquery.Builder(new NetHttpTransport(), new JacksonFactory(), credential)
.setApplicationName("name").build();

List rowList = new ArrayList();

TableRow rowWrapper = new TableRow();
rowWrapper.set("date", item.GetDate());
rowWrapper.set("value", item.GetValue());

TableDataInsertAllRequest.Rows insertRowRequest = new TableDataInsertAllRequest.Rows();

rowList.add(insertRowRequest);

TableDataInsertAllRequest content = new TableDataInsertAllRequest().setRows(rowList);

InsertAll op = bigquery.tabledata().insertAll("project", "dataset", "table", content);

TableDataInsertAllResponse response = op.execute();
if (response.getInsertErrors() == null)
{
; // all good
}
// example

贡献指南

打开贡献指南

调研方向

The report names Bigquery.tabledata().insertAll and TableDataInsertAllResponse.getInsertErrors(), but no repository file or test. Start by building a minimal high-throughput reproduction against BigQuery and inspecting the insert response behavior; done means the failure is isolated with evidence or the issue is documented as unreproducible.

由索引模型根据 Issue 内容生成。

评估

技术栈
google-cloud, java
领域
cloud, databases
Issue 类型
缺陷
难度
4/5
预计耗时
3-5 天
活跃度
停滞
描述清晰度
需要澄清
新手友好度
25/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。