Collision detection fails

This tends to happen when a bullet passes through an asteroid and the asteroid moves just enough so the line that is made from the current and last position of the asteroid is fully inside him.

A quick fix for this that wouldn't cover the edge cases is to do a radius check between the bullet and the asteroid with a circle that is fully encompassed by the asteroid so the bullet can't just pass through the center.

I don't see a better way atm since it doesn't happen every time and depends mostly on luck.

This is what I think the problem is but with some asteroids I never get a collision even if I hit them dead center.