NVIDIA / NVIDIA/cudf

[BUG] OOM when invoking normalize_characters on a relatively small dataframe

Open
#10,858 9 comments 0 reactions 0 assignees View on GitHub
0 - Backlog bug libcudf Python strings
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

**Describe the bug**
Hi, I am facing the following OOM when invoking normalize_characters() on a relatively small dataset, which uses less than 2 GB of VRAM on a 16GB v100.

In case it helps, normalize_spaces() works without issues in the same dataframe.

Please, find below how to reproduce it.

![image](https://user-images.githubusercontent.com/26169771/168416343-b712b4bf-e92b-44b4-babb-6f303130a87e.png)

**Steps/Code to reproduce bug**

> import cudf
> import pandas as pd
>
> df = cudf.DataFrame({"text": pd.util.testing.rands_array(256, 5000000)})
> #df["text"] = df["text"].str.normalize_spaces()
> df["text"] = df["text"].str.normalize_characters(do_lower=False)
>

**Expected behavior**
No OOM

**Environment overview (please complete the following information)**
- Environment location: [Bare-metal, Docker, Cloud(specify cloud provider)]
- Method of cuDF install: [conda, Docker, or from source]
- If method of install is [Docker], provide `docker pull` & `docker run` commands used

**Environment details**
DGX-v100, cudf 22.04

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.