alibaba / alibaba/FederatedScope

one bug in federatedscope/gfl/fedsageplus/trainer.py

Open
#767 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.5k
Forks
261
PR merge metrics
No merged PRs in 30d

Description

In the trainer.py,
`ctx.y_true = batch.num_missing[mask]`
`ctx.y_prob = pred_missing`

https://github.com/alibaba/FederatedScope/blob/480b67de851df2fa02b7cc2189f1803d947998d7/federatedscope/gfl/fedsageplus/trainer.py#L42C1-L43C34

https://github.com/alibaba/FederatedScope/blob/480b67de851df2fa02b7cc2189f1803d947998d7/federatedscope/gfl/fedsageplus/trainer.py#L66C1-L67C34

It seems that 'pred_missing' (i.e., the output of the missing neighbor generator) and 'num_missing' (i.e., the groundtruth of missing nodes) are used to compute the final result. However, it is a node classification task. We should compute the node classification accuracy.

As a result, I suggest that the node label and the predicted node label should be used to compute the final result.

It may be changed to
`ctx.y_true = batch.y[mask]`
`ctx.y_prob = nc_pred`

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading federatedscope/gfl/fedsageplus/trainer.py at lines 42-43 and 66-67. Compare the missing-node values with the node-label and prediction values described in the issue, then verify that the final result reports node classification accuracy rather than missing-node prediction performance.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.