NVIDIA-Merlin / NVIDIA-Merlin/Merlin
[BUG] FileNotFoundError when apply Categorify after JoinExternal
Open
Nobody has claimed this yet.
bug
- Dominant language
- Python
- Stars
- 907
- Forks
- 129
- PR merge metrics
- No merged PRs in 30d
Description
Bug description
When I try to use the Categorify() operator on a column after a JoinExternal(), I have the following kind of error:
FileNotFoundError: [Errno 2] No such file or directory: '~/data/categories/unique.XXXXX.parquet'
Expected behavior
Save file unique values of categorified column, like the ones done before the JoinExternal operation.
Environment details
- Merlin version: 23.8.0
- Platform: macOS
- Python version: 3.11.6
- Tensorflow version (GPU?): 2.12.1 CPU
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the Categorify() operation on a column after JoinExternal() with the reported Python and Merlin versions. Trace how Categorify saves unique values after the JoinExternal entry point and verify that the expected unique.XXXXX.parquet file is created at the configured path.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100