citusdata / citusdata/citus

Converting table to columnar storage fails with out of memory error

Open
#7,199 1 comment 1 reaction 0 assignees View on GitHub
columnar
Dominant language
C
Stars
12.8k
Forks
794
Avg merge
2d 14h
Merged PRs (30d)
31

Description

So the detailed error is as follows:
```
ERROR: out of memory
DETAIL: Cannot enlarge string buffer containing 1073253624 bytes by 1269320 more bytes.
CONTEXT: SQL statement "INSERT INTO public.raw_p2020_03_2430651750 (id,time,raw) OVERRIDING SYSTEM VALUE SELECT id,time,raw FROM public.raw_p2020_03"
SQL statement "SELECT pg_catalog.alter_table_set_access_method(r.partition, new_access_method)"
PL/pgSQL function alter_old_partitions_set_access_method(regclass,timestamp with time zone,name) line 27 at PERFORM
```

The problem partition has about 14000 large text blobs that in total are about 8.5GB (range 1.6 MB to 150kB in individual size) and this causes citus_columnar to fail the conversion. But we really like the compression citus can provide in columnar mode for this text (lost of very easy to compress text in those values)

So we set the stripe size to it's lowest possible setting (1000) but this is still not enough. Is there any way to fix this? Can citus automatically make smaller stripes when about to hit this limit?

We'd also get this error if we ran a COPY command to ingest into a table that was already columnar so it seems to be inserting/encoding related.

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the failure during alter_table_set_access_method and COPY with large text values, using the 1,000-row stripe setting described here. Trace alter_old_partitions_set_access_method and the columnar encoding path to identify where the string buffer exceeds its limit. Done means conversion and ingestion handle these values without an out-of-memory error while retaining columnar compression.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, postgresql
Domain
databases
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.