citusdata / citusdata/cstore_fdw
Skip indexes are not in use
- Dominant language
- C
- Stars
- 1.8k
- Forks
- 169
- PR merge metrics
- No merged PRs in 30d
Description
Hey, I am trying to tame columnar store in our enterprise solution and faced this behaviour
select org, sum(saleit), sum(costit)
from sale_agg_cs_top
where
dt = '2021-01-01'::date
and it = 0
group by org
execution plan would be
GroupAggregate (cost=214852.39..214859.59 rows=270 width=68) (actual time=596.516..597.755 rows=981 loops=1)
Group Key: org
Buffers: shared hit=22080
-> Sort (cost=214852.39..214853.17 rows=315 width=13) (actual time=596.500..596.593 rows=995 loops=1)
Sort Key: org
Sort Method: quicksort Memory: 71kB
Buffers: shared hit=22080
-> Foreign Scan on sale_agg_cs_top (cost=0.00..214839.32 rows=315 width=13) (actual time=1.652..595.635 rows=995 loops=1)
Filter: ((dt = '2021-01-01'::date) AND (it = 0))
Rows Removed by Filter: 4362836
CStore File: /var/lib/pgsql/10_5432/data/cstore_fdw/16409/197166589
CStore File Size: 263228026
Buffers: shared hit=22080
Planning time: 0.126 ms
Execution time: 598.100 ms
but when I involve a join
with fact as (
select org, sum(saleit), sum(costit)
from sale_agg_cs_top
where
dt = '2021-01-01'::date
and it = 0
group by org
)
select f.org, ka."JustDescription", f.*
from
fact as f
join
"TableDescription" as ka
on ka.org = f.org
I encounter a great degradation
Nested Loop (cost=214859.86..215160.24 rows=270 width=101) (actual time=1799.279..1804.731 rows=981 loops=1)
Buffers: shared hit=25023
CTE fact
-> GroupAggregate (cost=214852.39..214859.59 rows=270 width=68) (actual time=1799.247..1800.919 rows=981 loops=1)
Group Key: sale_agg_cs_top.org
Buffers: shared hit=22080
-> Sort (cost=214852.39..214853.17 rows=315 width=13) (actual time=1799.213..1799.333 rows=995 loops=1)
Sort Key: sale_agg_cs_top.org
Sort Method: quicksort Memory: 71kB
Buffers: shared hit=22080
-> Foreign Scan on sale_agg_cs_top (cost=0.00..214839.32 rows=315 width=13) (actual time=6.196..1798.565 rows=995 loops=1)
Filter: ((dt = '2021-01-01'::date) AND (it = 0))
Rows Removed by Filter: 13497703
CStore File: /var/lib/pgsql/10_5432/data/cstore_fdw/16409/197166589
CStore File Size: 263228026
Buffers: shared hit=22080
-> CTE Scan on fact f (cost=0.00..5.40 rows=270 width=68) (actual time=1799.253..1801.392 rows=981 loops=1)
Buffers: shared hit=22080
-> Index Scan using "pКонтрагент" on "Контрагент" ka (cost=0.28..1.09 rows=1 width=33) (actual time=0.003..0.003 rows=1 loops=981)
Index Cond: ("@Лицо" = f.org)
Buffers: shared hit=2943
Planning time: 1.960 ms
Execution time: 1805.475 ms
As you can see first time it scaned 4362836 and second time 13497703. The table sale_agg_cs_top has 13498698 rows. That's why I think skip indexes are not in use.
All tables were analyzed.
Versions
PostgreSQL 10.15 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-39), 64-bit
cstore_fdw 1.7
CentOS 7.7.1908
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reproducing the two SQL queries on PostgreSQL 10.15 with cstore_fdw 1.7, then compare their foreign-scan plans and row-filtering counts. Investigate why the join changes the scan behavior and verify the fix by confirming that skip indexes are used consistently in both plans.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c, postgresql
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100