awslabs / awslabs/graphstorm

Using numpy.array to store string object is not memory efficient.

Open
#775 0 comments 0 reactions 0 assignees View on GitHub
efficiency bug
Dominant language
Python
Stars
452
Forks
76
PR merge metrics
No merged PRs in 30d

Description

https://github.com/awslabs/graphstorm/blob/main/python/graphstorm/gconstruct/file_io.py#L153
The default behavior of numpy.array(string) will allocate too much memory than required, which is not efficient.

Contributor guide

Open the contributing guide

Research direction

Start by reading python/graphstorm/gconstruct/file_io.py at line 153 and inspect how numpy.array stores the string object. Compare the current allocation with a more memory-efficient representation, then verify that string handling remains correct and that the reported excess memory is reduced.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, python
Domain
machine-learning
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.