CachedNetworkImage makes button not respond to clicks when setting fadeOutDuration to zero

## 🐛 Bug Report

I'm using `CachedNetworkImage` with the `imageBuilder` property, like showed in the example.
Down the widget tree I had a button and I noticed it's `onPressed` method was never called and it appeared to be stuck.
After some trial and error I've discovered that it is caused by using `CachedNetworkImage` with the `fadeOutDuration` set to `Duration.zero`. It is worth noting that even if I wait, the button never becomes active.

### Expected behavior
The button should still be clickable.

### Reproduction steps
- Create a `CachedNetworkImage` with an `imageBuilder` and `fadeOutDuration` set to `Duration.zero`.
- Add a button inside the image builder.

I'm adding a code sample:
```
CachedNetworkImage(
imageUrl: "https://www.applesfromny.com/wp-content/uploads/2020/06/SnapdragonNEW.png",
fadeOutDuration: Duration.zero,
placeholder: (context, url) => const CircularProgressIndicator(),
imageBuilder: (context, imageProvider) => TextButton(
onPressed: () => print("clicked"),
child: const Icon(
Icons.edit,
color: Colors.white,
),
),
);
```

### Configuration
Flutter: 3.3.10
Happens on android emulator, did not test on a real device.

**Version:** 3.2.1

**Platform:**
- [ ] :iphone: iOS
- [x] :robot: Android