ScrollAction slows down significantly when holding down keys

### Steps to reproduce

1. Place a [`MaterialApp`](https://api.flutter.dev/flutter/material/MaterialApp-class.html) widget at the root of your widget tree.
2. Place any scrollable widget below the [`MaterialApp`](https://api.flutter.dev/flutter/material/MaterialApp-class.html) and make sure it uses the [`PrimaryScrollController`](https://api.flutter.dev/flutter/widgets/PrimaryScrollController-class.html).
3. Try scrolling down by pressing ctrl+down for desktop or just the down arrow for web.
4. Then try doing the same, but holding the keys down.

### Expected results

I would expect to scroll faster when holding down the keys than when pressing them repeatedly.

### Actual results

Scrolling becomes significantly slower when holding down the keys compared to pressing them repeatedly.

### Code sample

Code sample

```dart
import 'package:flutter/material.dart';

void main() => runApp(const ExampleApp());

class ExampleApp extends StatelessWidget {
const ExampleApp({super.key});

@override
Widget build(BuildContext context) => MaterialApp(
home: Scaffold(
body: ListView.builder(
primary: true,
itemCount: 15,
itemBuilder: (BuildContext context, int index) => Container(
color: Colors.primaries[index],
height: 150,
child: Center(
child: Text(
'Entry $index',
style: Theme.of(context).textTheme.headlineMedium,
),
),
),
),
),
);
}
```

### Screenshots or Video

Screenshots / Video demonstration

The first part shows the behavior when pressing the scroll keys repeatedly, and then while holding the scroll keys down.


### Logs

_No response_

### Flutter Doctor output

Doctor output

```console
[✓] Flutter (Channel stable, 3.24.5, on Fedora Linux 41 (Workstation Edition) 6.11.7-300.fc41.x86_64, locale
en_US.UTF-8)
• Flutter version 3.24.5 on channel stable at /home/kamil/.local/apps/flutter
• Upstream repository https://github.com/flutter/flutter.git
• Framework revision dec2ee5c1f (7 days ago), 2024-11-13 11:13:06 -0800
• Engine revision a18df97ca5
• Dart version 3.5.4
• DevTools version 2.37.3

[✗] Android toolchain - develop for Android devices
✗ Unable to locate Android SDK.
Install Android Studio from: https://developer.android.com/studio/index.html
On first launch it will assist you in installing the Android SDK components.
(or visit https://flutter.dev/to/linux-android-setup for detailed instructions).
If the Android SDK has been installed to a custom location, please use
`flutter config --android-sdk` to update to that location.

[✓] Chrome - develop for the web
• CHROME_EXECUTABLE = /usr/bin/brave-browser-stable

[✓] Linux toolchain - develop for Linux desktop
• clang version 19.1.0 (Fedora 19.1.0-1.fc41)
• cmake version 3.30.5
• ninja version 1.12.1
• pkg-config version 2.3.0

[!] Android Studio (not installed)
• Android Studio not found; download from https://developer.android.com/studio/index.html
(or visit https://flutter.dev/to/linux-android-setup for detailed instructions).

[✓] VS Code (version 1.95.3)
• VS Code at /usr/share/code
• Flutter extension version 3.100.0

[✓] Connected device (2 available)
• Linux (desktop) • linux • linux-x64 • Fedora Linux 41 (Workstation Edition) 6.11.7-300.fc41.x86_64
• Chrome (web) • chrome • web-javascript • Brave Browser 131.1.73.89

[✓] Network resources
• All expected network resources are available.

! Doctor found issues in 2 categories.
```