duckdb / duckdb/duckdb-httpfs

HTTP stats shown even for queries not involving HTTP requests

Open
#294 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
60
Forks
100
Avg merge
1h 50m
Merged PRs (30d)
25

Description

Once the HTTPFS stats are shown in `EXPLAIN ANALYZE` output, the box is shown in subsequent outputs even though the queries do not involve any HTTP request. I think the stats should show only for queries involving HTTP requests.

To reproduce:

```sql
EXPLAIN ANALYZE SELECT 1;
```

Output doesn't show the stats initially:

```
┌─────────────────────────────────────┐
│┌───────────────────────────────────┐│
││ Query Profiling Information ││
│└───────────────────────────────────┘│
└─────────────────────────────────────┘
EXPLAIN ANALYZE SELECT 1;
┌────────────────────────────────────────────────┐
│┌──────────────────────────────────────────────┐│
││ Total Time: 0.0005s ││
│└──────────────────────────────────────────────┘│
└────────────────────────────────────────────────┘
┌───────────────────────────┐
│ QUERY │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│ EXPLAIN_ANALYZE │
│ ──────────────────── │
│ │
│ 0 rows │
│ 0.00s │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│ PROJECTION │
│ ──────────────────── │
│ 1 │
│ │
│ │
│ │
│ 1 row │
│ 0.00s │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│ DUMMY_SCAN │
│ ──────────────────── │
│ │
│ 1 row │
│ 0.00s │
└───────────────────────────┘
```

Now perform a query involving HTTP request:

```sql
EXPLAIN ANALYZE SELECT count(*) FROM 'https://community-extensions.duckdb.org/downloads-last-week.json';
```

Output contains HTTP stats:

```
┌─────────────────────────────────────┐
│┌───────────────────────────────────┐│
││ Query Profiling Information ││
│└───────────────────────────────────┘│
└─────────────────────────────────────┘
EXPLAIN ANALYZE SELECT count(*) FROM 'https://community-extensions.duckdb.org/downloads-last-week.json';
┌─────────────────────────────────────┐
│┌───────────────────────────────────┐│
││ HTTPFS HTTP Stats ││
││ ││
││ in: 3.0 KiB ││
││ out: 0 bytes ││
││ #HEAD: 1 ││
││ #GET: 1 ││
││ #PUT: 0 ││
││ #POST: 0 ││
││ #DELETE: 0 ││
│└───────────────────────────────────┘│
└─────────────────────────────────────┘
┌────────────────────────────────────────────────┐
│┌──────────────────────────────────────────────┐│
││ Total Time: 0.185s ││
│└──────────────────────────────────────────────┘│
└────────────────────────────────────────────────┘
┌───────────────────────────┐
│ QUERY │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│ EXPLAIN_ANALYZE │
│ ──────────────────── │
│ │
│ 0 rows │
│ 0.00s │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│ UNGROUPED_AGGREGATE │
│ ──────────────────── │
│ Aggregates: │
│ count_star() │
│ │
│ │
│ │
│ 1 row │
│ 0.00s │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│ TABLE_SCAN │
│ ──────────────────── │
│ Function: │
│ READ_JSON_AUTO │
│ │
│ Total Files Read: 1 │
│ │
│ Filename(s): │
│ https://community │
│ -extensions.duckdb.org │
│ /downloads-last-week.json │
│ │
│ │
│ │
│ 1 row │
│ 0.00s │
└───────────────────────────┘
```

Now performing the original query `EXPLAIN ANALYZE SELECT 1` shows HTTP stats with empty counters:

```
┌─────────────────────────────────────┐
│┌───────────────────────────────────┐│
││ Query Profiling Information ││
│└───────────────────────────────────┘│
└─────────────────────────────────────┘
EXPLAIN ANALYZE SELECT 1;
┌─────────────────────────────────────┐
│┌───────────────────────────────────┐│
││ HTTPFS HTTP Stats ││
││ ││
││ in: 0 bytes ││
││ out: 0 bytes ││
││ #HEAD: 0 ││
││ #GET: 0 ││
││ #PUT: 0 ││
││ #POST: 0 ││
││ #DELETE: 0 ││
│└───────────────────────────────────┘│
└─────────────────────────────────────┘
┌────────────────────────────────────────────────┐
│┌──────────────────────────────────────────────┐│
││ Total Time: 0.0012s ││
│└──────────────────────────────────────────────┘│
└────────────────────────────────────────────────┘
┌───────────────────────────┐
│ QUERY │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│ EXPLAIN_ANALYZE │
│ ──────────────────── │
│ │
│ 0 rows │
│ 0.00s │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│ PROJECTION │
│ ──────────────────── │
│ 1 │
│ │
│ │
│ │
│ 1 row │
│ 0.00s │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│ DUMMY_SCAN │
│ ──────────────────── │
│ │
│ 1 row │
│ 0.00s │
└───────────────────────────┘
```

I might try to prepare a PR that would show the stats in `EXPLAIN ANALYZE` conditionally.

Contributor guide

No contributing guide indexed for this repository

Research direction

Reproduce the behavior with the two EXPLAIN ANALYZE queries shown in the issue, first using SELECT 1 and then the HTTP URL query. Trace how HTTPFS HTTP Stats are included in EXPLAIN ANALYZE output; done means HTTP stats appear for HTTP requests but not for later queries that perform no HTTP requests, with empty counters omitted.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
networking
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.