vectordotdev / vectordotdev/vector
Improve error logging in `gcp_cloud_storage` sink
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 22.6k
- Forks
- 2.3k
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 146
Description
From chat by @alathon:
I'm getting a '400 Bad Request' from GCP on my GCS sink, but even on TRACE level it's not showing the body of the response so I can't get at which actual problem it's encountering. All the output I get on trace is:
Apr 07 13:50:06.853 TRACE sink{name=gcp type=gcp_cloud_storage}: vector::sinks::util: request succeeded. response=Response { status: 400, version: HTTP/1.1, headers: {"x-guploader-uploadid": "xxx", "content-type": "application/xml; charset=UTF-8", "content-length": "170", "vary": "Origin", "date": "Tue, 07 Apr 2020 13:50:06 GMT", "server": "UploadServer", "alt-svc": "quic=\":443\"; ma=2592000; v=\"46,43\",h3-Q050=\":443\"; ma=2592000,h3-Q049=\":443\"; ma=2592000,h3-Q048=\":443\"; ma=2592000,h3-Q046=\":443\"; ma=2592000,h3-Q043=\":443\"; ma=2592000,h3-T050=\":443\"; ma=2592000"}, body: Body(Streaming) }The
bodyproperty there doesn't get revealed further down in the log anywhere, and then the connection closes.
It seems like the http connection is being closed by the caller before the body can be received fully? :sApr 07 14:05:54.397 TRACE hyper::proto::h1::conn: State::close_read() Apr 07 14:05:54.397 TRACE hyper::proto::h1::conn: State::close() Apr 07 14:05:54.397 TRACE tokio_threadpool::worker: -> wakeup; idx=3 Apr 07 14:05:54.397 TRACE hyper::proto::h1::conn: flushed({role=client}): State { reading: Closed, writing: Closed, keep_alive: Disabled }```
I'm not sure is the issue GCP-specific or should it be addressed for all HTTP-based sinks.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the gcp_cloud_storage sink and tracing how its HTTP response is logged at TRACE, using the reported 400 Bad Request scenario. Done means the relevant response body is available in logs, with the scope clarified between GCP-specific handling and all HTTP-based sinks.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- google-cloud, rust
- Domain
- cloud, observability
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100