alibaba / alibaba/FederatedScope

When is the serializer used? Why for training only but not eval

Open
#790 0 comments 0 reactions 2 assignees Claimed by @xieyxclack View on GitHub
Dominant language
Python
Stars
1.5k
Forks
261
PR merge metrics
No merged PRs in 30d

Description

I noticed the data processing stage is extremely slow in eval stage. It seems that its caused by not serializing the params.

In training stage msg_type is 'model_para', while in eval stage it's 'evaluate'. But model params are transmitted in both stages.

federatedscope/core/message.py line 106 - 120:


def transform_to_list(self, x):

if isinstance(x, list) or isinstance(x, tuple):

return [self.transform_to_list(each_x) for each_x in x]

elif isinstance(x, dict):

for key in x.keys():

x[key] = self.transform_to_list(x[key])

return x

else:

if hasattr(x, 'tolist'):

if self.msg_type == 'model_para':

return self.param_serializer(x)

else:

return x.tolist()

else:

return x

We can see params are serialized in training stage, but not in other messages. I'm curious why it's designed like that, I’d really appreciate it if you could help me with this.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.