[OP] Add dropDuplicates() operator
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 35/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Stale
- Tech stack
- python
- Domain
- data, machine-learning
Research direction
The issue names no repository file or test; start by locating existing operator implementations and their tests, then compare the requested behavior with the referenced cuDF drop_duplicates usage. Done means the operator accepts continuous and categorical columns and returns a DataFrame with duplicate rows removed, keeping the first occurrence.
Written by the indexing model from the issue text.
Description
Tuesday May 26, 2020 at 18:00 GMT
Originally opened as https://github.com/rapidsai/recsys/issues/172
Is your operator request related to a problem? Please describe.
dropDuplicates() method is used in the Outbrain W&D model, and it is one of the commonly used methods in data preprocessing.
Describe the solution you'd like
A clear and concise description of the operation you'd like to perform on the column. Please include:
- Type (Feature Engineering or Preprocessing): Preprocessing
- input column type(s): Continuous and categorical
- output column type(s): Continuous and categorical
- Expected transformation of the data after application: Return DataFrame with duplicate rows removed.
Additional context
cudf has dropDuplicates() method and applied as below:
cdf.drop_duplicates(keep= 'first', inplace=True)
- Dominant language
- Python
- Stars
- 1.2k
- Forks
- 149
- PR merge metrics
- No merged PRs in 30d
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from NVIDIA-Merlin/NVTabular
-
[DOC] Open
Difficulty 4/5 3-5 days Newbie friendliness 20/100
NVIDIA-Merlin/NVTabular#1898 · 1 comment ·
-
question
Difficulty 5/5 Over a week Newbie friendliness 25/100
NVIDIA-Merlin/NVTabular#1895 ·
-
question
Difficulty 4/5 3-5 days Newbie friendliness 25/100
NVIDIA-Merlin/NVTabular#1893 ·
-
question
Difficulty 5/5 Over a week Newbie friendliness 20/100
NVIDIA-Merlin/NVTabular#1892 · 1 reaction ·
-
question
Difficulty 4/5 3-5 days Newbie friendliness 35/100
NVIDIA-Merlin/NVTabular#1891 ·
All issues in NVIDIA-Merlin/NVTabular
Similar issues
-
area/auth bug comp/agent P3 platform/discord type/security
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
NousResearch/hermes-agent#117848 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
bancolombia/sentinel#23 ·
-
test md OpenCI
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
-
integration:quickjs org:external priority:backlog topic:code-interpreter topic:middleware type:feature
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
langchain-ai/deepagents#6450 ·
-
bug client
Difficulty 2/5 1-3 hours Newbie friendliness 88/100