controller copyWith not effective in fullscreen

I am changing the overlay when the video is playing. I added a listener to my controller and in it, I check the isPlaying property. If the video is playing, I change the overlay by updating the controller using the copyWith method (in a setState).
It works well when the video is not in fullscreen.

When in fullscreen, the overlay doesn't get changed, although isPlaying is true and the line where I update the controller gets executed.