When cache_httpfs is loaded, HTTP response headers (e.g., Content-Range) are not captured in HTTP logging (duckdb_logs_parsed('HTTP'))
- Dominant language
- C++
- Stars
- 60
- Forks
- 100
- Avg merge
- 1h 50m
- Merged PRs (30d)
- 25
Description
**Description:**
When cache_httpfs is loaded, HTTP response headers (e.g., Content-Range) are not captured in HTTP logging (duckdb_logs_parsed('HTTP')).
**Root Cause:**
cache_httpfs registers a CacheFileSystem that wraps httpfs. On cache miss, it calls internal_filesystem->Read() directly on the httpfs file handle. This invokes HTTPFileSystem::ReadInternal → GetRangeRequest which performs the HTTP GET, but the streaming response path in httpfs_httplib_client.cpp does not propagate response headers to the final HTTPResponse object that LogRequest receives.
**Code path comparison:**
Without cache_httpfs (works):
HTTPFileSystem::OpenFile → HTTPUtil::RunRequestWithRetry → LogRequest(request, response) ✓
With cache_httpfs (broken):
CacheFileSystem::Read → DiskCacheReader::ReadAndCache → internal_filesystem->Read()
→ HTTPFileSystem::ReadInternal → GetRangeRequest → http_util.Request()
→ Response headers available in streaming callback but not in final response object passed to LogRequest ✗
**Impact:**
- request.headers['Range'] is logged correctly
- response.headers['Content-Range'] is NULL in logs
**Suggested fix:**
Ensure response headers from the streaming on_response callback in GetRangeRequest are preserved in the HTTPResponse object returned by http_util.Request(), so LogRequest can log them.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start in httpfs_httplib_client.cpp and trace GetRangeRequest through http_util.Request() to LogRequest, comparing the streaming on_response callback with the final HTTPResponse. Reproduce the cache_httpfs path with HTTP logging and verify that response.headers['Content-Range'] is populated while request.headers['Range'] remains logged.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- networking, observability
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 72/100