Clarification for # Active Experts

Hi,

Just another quick question, sorry. For inference, the paper states there are 4 out of 8 experts active, but the [allenai/FlexOlmo-7x7B-1T config](https://huggingface.co/allenai/FlexOlmo-7x7B-1T/blob/main/config.json) shows 5 out of 7 experts, and the [allenai/FlexOlmo-7x7B-1T-RT config](https://huggingface.co/allenai/FlexOlmo-7x7B-1T-RT/blob/main/config.json) shows 7 out of 7 experts active. Should I use 4 from the paper? And out of curiosity, can you explain how this number was chosen?

Also, I'm assuming there was originally 8 experts but it was changed to release as 7 experts? Thanks!

Also, during training of each expert, was it using a top-1 hard routing?

Thank you guys so much!