[Python][CI] Simplify requirements-wheel-test.txt

The current requirements-wheel-test.txt file has complex version pins for numpy and pandas depending on the python version and platform.

I think we can simplify this to just pin numpy based on the python version (to ensure we are still testing with some older version of numpy, and not the latest, so that this would fail if we accidentally built the wheel with the latest numpy).
And the pins for pandas can be removed entirely (we don't have any build dependency on pandas, and we test older pandas versions in other builds)