Enhance Lyra Quality on μ-Law Decoded Audio Stream

I am currently working with Lyra on an audio stream that has been decoded from μ-law (u-law) encoding for VoIP application. Maybe because Lyra was not trained on μ-law decoded samples, the quality of the output, assessed both by listening and using the PESQ test, results in a mean score of approximately 2.4 for the 3200 bitrate mode (It's even less than codec2 results on same input 😭). I am seeking ways to improve the quality of the results.

Current Setup:

Input: μ-law encoded audio stream.
Decoding: μ-law to signed 16 bit PCM using audioop.
Lyra-Encoding: Lyra for further processing.
--Sending Lyra-encoded packets--
Lyra-Decoding.
μ-law Encoding.

Challenges:
The decoded audio lacks clarity, resulting in suboptimal PESQ scores.
The audio output could be more intelligible and natural (at the moment the audio output sounds a bit hoarse.).

Request for Suggestions:
I am looking for advice or techniques that could help improve the quality of the audio output, potentially through:
Adjustments to the u-law decoding process.
Any other relevant enhancements to the Lyra configuration or processing.
Thank you for your help! 🙌