Example differentiability-leveraging algorithm parameters for legged systems?

Has anyone had any success with differentiability-leveraging learning algorithms (e.g. the built-in APG) for legged systems, like HalfCheetah or Ant? I see that there's an example piece of code for the contact-less reacher, but that's it. Does anyone have example hyperparameters for the aforementioned two systems? When I try running it, oftentimes I get NaNs/infinities in the weights. Any help would be appreciated!