redpanda-data / redpanda-data/connect
GCP cloud storage output error with Failed to delete temporary file used for merging: context canceled
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 8.8k
- Forks
- 969
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 64
Description
Context
I am reading bunch of event coming from a MQTT broker
input:
label: "benthos_mqtt_test"
mqtt:
urls:
- localhost:1883
client_id: benthos-reader
connect_timeout: 30s
topics:
- $share/benthos/XXXX/#
and I am trying to process and save the output into blobstorage (GCP cloud storage).
The processing is relatively simple, I am taking an mqtt message (slice of events) and outputting to a CSV like format
buffer:
none: {}
pipeline:
threads: -1
processors:
- unarchive:
format: "json_map"
- bloblang: |
let csv = match {
this.value.type() == "bool" => ",,"+this.value.string()
this.value.type() == "number" => this.value.string()+",,"
_ => ","+this.value.string()+","
}
root = this.ts.ts_strftime("%Y-%m-%dT%H:%M:%S%z").string() + "," + meta("archive_key") + "," + $csv +"\n"
meta dt = this.ts.ts_strftime("%Y-%m-%d", "UTC")
meta id = meta("archive_key").escape_url_query()
the output logic is to create folder partition by date (dt=YYYY/MM/DD) and id (id=ID)
output:
label: "blobstorage"
#stdout: {}
gcp_cloud_storage:
bucket: "abucket"
path: benthos/dt=${!@dt}/id=${!@id}/file.csv
collision_mode: append
timeout: 60s
max_in_flight: 64
when running Benthos with the config, I am seeing quite often this error
ERRO Failed to delete temporary file used for merging: context canceled @service=benthos label=blobstorage path=root.output
I am not 100% sure if this is false positive since I have quite a lot of event comming per sec (~1000 events) and the error doesn't tell me much about the path timing out so I cannot really check if the temp file has been merged or not.
i tried to change timeout 20s, 30s 60s but seems to always error.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the gcp_cloud_storage output configuration and its collision_mode: append behavior, using the supplied MQTT pipeline and high-throughput settings to reproduce the context-canceled deletion error. Check whether the temporary file was merged despite the error and identify which timeout or cancellation path produces the message. Done means documenting the cause and making the error outcome unambiguous.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- gcp, go
- Domain
- cloud, data-engineering
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100