Bug in LlamaMLP layer to detect neurons (inside Neuron Detection folder)

Is not it bug in the given line https://github.com/DAMO-NLP-SG/multilingual_analysis/blob/main/neuron_detection/transformers/models/llama/modeling_llama.py#L246

`return real_output, torch.sum(torch.abs(self.up_proj(x)), dim=1).squeeze().tolist(), torch.sum(torch.abs(self.up_proj(x)), dim=1).squeeze().tolist()`

Should it not be self.up_proj(x) and self.down_proj(x)?

Can authors please clarify this?

Start at neuron_detection/transformers/models/llama/modeling_llama.py line 246 and trace the LlamaMLP outputs used for neuron detection. Compare the roles of up_proj(x) and down_proj(x), then verify whether the duplicate expression is intentional; done means the projection outputs and their returned neuron measurements are correct.

Written by the indexing model from the issue text.