Speech V2 interim_results rpc is blocking

My code is very similar to the official documentation snippet [transcribe_streaming_v2.py](https://github.com/GoogleCloudPlatform/python-docs-samples/blob/main/speech/snippets/transcribe_streaming_v2.py)

but I am using `interim_results=True` because I am getting audio from streaming source, so my config looks like this:
```python
recognition_config = cloud_speech.RecognitionConfig(
auto_decoding_config=cloud_speech.AutoDetectDecodingConfig(),
language_codes=["en-US"],
model="long",
)
streaming_features = cloud_speech.StreamingRecognitionFeatures(interim_results=True)
streaming_config = cloud_speech.StreamingRecognitionConfig(
config=recognition_config,
streaming_features=streaming_features
)
config_request = cloud_speech.StreamingRecognizeRequest(
recognizer=f"projects/{project_id}/locations/global/recognizers/_",
streaming_config=streaming_config,
)

def requests(config: cloud_speech.RecognitionConfig, audio: list) -> list:
yield config
yield from audio

# Transcribes the audio into text
responses_iterator = client.streaming_recognize(
requests=requests(config_request, audio_requests)
)
```
Now the `responses_iterator` should return right away with a generator to be iterated over with the text inside, but it doesn't do that until all requests are done and raises an error instead:
```python
...
...
File "/home/admin/realtime_transcriptions/.venv/lib/python3.11/site-packages/google/cloud/speech_v2/services/speech/client.py", line 1639, in streaming_recognize
response = rpc(
^^^^
File "/home/admin/realtime_transcriptions/.venv/lib/python3.11/site-packages/google/api_core/gapic_v1/method.py", line 131, in __call__
return wrapped_func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/admin/realtime_transcriptions/.venv/lib/python3.11/site-packages/google/api_core/grpc_helpers.py", line 177, in error_remapped_callable
raise exceptions.from_grpc_error(exc) from exc
google.api_core.exceptions.Aborted: 409 Stream timed out after receiving no more client requests.
```

I tested this with many versions of `google-cloud-speech`. the speech V1 does not have this problem and works correctly with `interim_results=True` and returns the generator.
any idea if I am doing something wrong or this is a bug?