alibaba / alibaba/FederatedScope
When is the serializer used? Why for training only but not eval
- Dominant language
- Python
- Stars
- 1.5k
- Forks
- 261
- PR merge metrics
- No merged PRs in 30d
Description
I noticed the data processing stage is extremely slow in eval stage. It seems that its caused by not serializing the params.
In training stage msg_type is 'model_para', while in eval stage it's 'evaluate'. But model params are transmitted in both stages.
federatedscope/core/message.py line 106 - 120:
def transform_to_list(self, x):
if isinstance(x, list) or isinstance(x, tuple):
return [self.transform_to_list(each_x) for each_x in x]
elif isinstance(x, dict):
for key in x.keys():
x[key] = self.transform_to_list(x[key])
return x
else:
if hasattr(x, 'tolist'):
if self.msg_type == 'model_para':
return self.param_serializer(x)
else:
return x.tolist()
else:
return x
We can see params are serialized in training stage, but not in other messages. I'm curious why it's designed like that, I’d really appreciate it if you could help me with this.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.