alibaba / alibaba/GraphScope

Support Sparse Recommendation Model Training

Open
#2,642 0 comments 0 reactions 0 assignees View on GitHub
enhancement summer-of-code
Dominant language
C++
Stars
3.6k
Forks
468
Avg merge
29m
Merged PRs (30d)
1

Description

## Background

Graph learning is one of the major workloads supported by GraphScope. GNN's main application scenarios include recommendation systems, financial risk control, and biomedical development. Among them, GLE (the learning engine in GraphScope) has been widely applied in the scenario of recommendation. Most of the recommendation models are sparse model, where the size of embedding can reach the scale of tens of billions. To further integrate with the Pytorch ecosystem, we have launched [grephlearn-for-pytorch](https://github.com/alibaba/graphlearn-for-pytorch), a graph learning library for PyTorch that makes distributed GNN training and inference easy and efficient. When utilizing GPU for model training, grephlearn-for-pytorch can outperform GLE on end-to-end tasks by a significant margin. Additionally, the process of developing models is much easier compared to GLE, which is based on TensorFlow 1.x. We plan to build the next generation of GLE engine based on grephlearn-for-pytorch in GraphScope, thus supporting large-scale sparse model training on grephlearn-for-pytorch is very important.

[TorchRec](https://github.com/pytorch/torchrec) is a PyTorch library for recommendation systems. When building recommendation systems, it is often necessary to use embeddings to represent entities such as products or pages. As the number of entities grows, the size of the embedding table may exceed the memory of a single GPU. The common approach is to shard the embedding table, which is a model parallel method. TorchRec encapsulates a series of APIs to support model parallel training based on embedding sharding (DMP). In this project, we will implement the integration of grephlearn-for-pytorch with TorchRec and implement a distributed embedding module based on GraphSAGE for the recommendation GNN model.

Tasks:
- Support large-scale embedding in grephlearn-for-pytorch based on TorchRec.
- Properly design the APIs, provide a simple and user-friendly distributed solution for recommendation models.

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the grephlearn-for-pytorch and TorchRec integration requirements, focusing on embedding sharding and the proposed GraphSAGE-based distributed embedding module. Done means supporting large-scale embeddings and providing simple, user-friendly APIs for distributed recommendation model training.

Written by the indexing model from the issue text.

Assessment

Tech stack
pytorch
Domain
distributed-systems, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.