Camera's is_active changes slow down rendering

## Bevy version

v0.10.1

## What you did

I wanted to optimise [my code](https://github.com/Selene-Amanita/bevy_basic_portals/blob/346ad65df96cad375f37a87b6e6f88353c55f579/src/portals/update.rs#L71). I have several cameras rendering to several images used as texture of a CustomMaterial, for a portal effect.
I wanted to deactivate the cameras when their rendering is useless (the main camera doesn't look at the portal), and activate them again when needed.
Turns out, the app runs completely fine without that, but becomes jerky with it.

**I would expect deactivating a camera when not needed to actually makes performances better, but it makes them worse.**

I made a simpler example to test scenarios (and gifs bellow), it happens also if the cameras that are deactivated/activated render to a window, it's not noticeable (on my computer) with only one camera doing this.

## "Minimal" code to reproduce / play around with

If `.add_system(activate_and_deactivate_other_cameras)` is commented, the main camera sould turn around the cube and the cube appears to rotate smoothly.
If `.add_system(activate_and_deactivate_other_cameras)` is uncommented, and `CAMERA_COUNT` is high enough (>3 on my computer), the cube doesn't rotate smoothly anymore, you can see it "jump".
`SETUP` doesn't seem to change anything.
The slowing down happens in release mode too.

```rust
use bevy::{
prelude::*,
render::{
render_resource::*,
camera::RenderTarget
},
window::WindowRef
};

const SETUP: Setup = Setup::RenderToImage; //Epilepsy warning if you use RenderToWindow
const CAMERA_COUNT: u32 = 5;
const MAIN_CAMERA_ROTATION: f32 = 9.;
const ACTIVATE_FREQUENCY_INV: u128 = 500;

fn main() {
App::new()
.add_plugins(DefaultPlugins)
.add_startup_system(setup_camera_and_cube)
.add_startup_system(setup_other_cameras)
.add_system(rotate_main_camera)
//.add_system(activate_and_deactivate_other_cameras)
.run();
}

pub enum Setup {
RenderToImage,
RenderToMaterial,
RenderToWindow
}

#[derive(Component)]
struct MainCamera;

fn setup_camera_and_cube(
mut commands: Commands,
mut meshes: ResMut>,
mut images: ResMut>,
mut materials: ResMut>
) {
commands.insert_resource(AmbientLight {
color: Color::WHITE,
brightness: 0.3,
});

// Main camera (displayed in window and gets rotated)
commands.spawn((
Camera3dBundle {
transform: Transform::from_xyz(0.0, 0., 20.0).looking_at(Vec3::ZERO, Vec3::Y),
..default()
},
MainCamera
));

// Object
let sphere_mesh = meshes.add(Mesh::from(shape::Cube::new(5.)));
let debug_material = materials.add(StandardMaterial {
base_color_texture: Some(images.add(uv_debug_texture())),
..default()
});
commands.spawn(PbrBundle {
mesh: sphere_mesh,
material: debug_material,
..default()
});
}

fn setup_other_cameras(
mut commands: Commands,
mut meshes: ResMut>,
mut images: ResMut>,
mut materials: ResMut>
) {
// Other cameras get activated/deactivated, renders to an image or another window depending on SETUP
for i in 0..CAMERA_COUNT {
let target =
if let Setup::RenderToWindow = SETUP {
let window = commands.spawn(
Window::default()
).id();
RenderTarget::Window(WindowRef::Entity(window))
}
else {
let size = Extent3d {width: 450, height: 450, ..default()};
let mut camera_image = Image {
texture_descriptor: TextureDescriptor {
label: None,
size,
dimension: TextureDimension::D2,
format: TextureFormat::Bgra8UnormSrgb,
mip_level_count: 1,
sample_count: 1,
usage: TextureUsages::TEXTURE_BINDING
| TextureUsages::COPY_DST
| TextureUsages::RENDER_ATTACHMENT,
view_formats: &[],
},
..default()
};
camera_image.resize(size);
let camera_image = images.add(camera_image);

if let Setup::RenderToMaterial = SETUP {
let material_handle = materials.add(StandardMaterial {
base_color_texture: Some(camera_image.clone()),
reflectance: 0.02,
unlit: false,
..default()
});
commands.spawn((
PbrBundle {
mesh: meshes.add(Mesh::from(shape::Cube::new(5.))),
material: material_handle,
transform: Transform::from_xyz(0., 6.*(i as f32), 0.),
..default()
},
));
};

RenderTarget::Image(camera_image)
};

commands.spawn(
Camera3dBundle {
camera: Camera {
order: -1,
target,
..default()
},
transform: Transform::from_xyz(0., 0., 10.*(i as f32)),
..default()
}
);
}
}

fn rotate_main_camera(
time: Res,
mut camera_query: Query<&mut Transform, With>,
) {
let mut camera_transform = camera_query.single_mut();
camera_transform.rotate_around(Vec3::ZERO, Quat::from_axis_angle(Vec3::Y, time.delta_seconds() * MAIN_CAMERA_ROTATION))
}

fn activate_and_deactivate_other_cameras (
time: Res,
mut camera_query: Query<&mut Camera, Without>,
) {
for mut camera in camera_query.iter_mut() {
let time = time.elapsed().as_millis() % ACTIVATE_FREQUENCY_INV;
if time < (ACTIVATE_FREQUENCY_INV / 2) {
camera.is_active = true;
} else {
camera.is_active = false;
}
}
}

// (copied from bevy's 3d_shape example)
pub fn uv_debug_texture() -> Image {
const TEXTURE_SIZE: usize = 8;

let mut palette: [u8; 32] = [
255, 102, 159, 255, 255, 159, 102, 255, 236, 255, 102, 255, 121, 255, 102, 255, 102, 255,
198, 255, 102, 198, 255, 255, 121, 102, 255, 255, 236, 102, 255, 255,
];

let mut texture_data = [0; TEXTURE_SIZE * TEXTURE_SIZE * 4];
for y in 0..TEXTURE_SIZE {
let offset = TEXTURE_SIZE * y * 4;
texture_data[offset..(offset + TEXTURE_SIZE * 4)].copy_from_slice(&palette);
palette.rotate_right(4);
}

Image::new_fill(
Extent3d {
width: TEXTURE_SIZE as u32,
height: TEXTURE_SIZE as u32,
depth_or_array_layers: 1,
},
TextureDimension::D2,
&texture_data,
TextureFormat::Rgba8UnormSrgb,
)
}
```

## Gifs

It's better to test yourself but here's a capture of what it does, this is with `SETUP=RenderToImage`, and `CAMERA_COUNT=10`

Without `activate_and_deactivate_other_cameras`:

With `activate_and_deactivate_other_cameras`:

## Another test

If you clone [bevy_basic_portals](https://github.com/Selene-Amanita/bevy_basic_portals), and run the cube example (`cargo run --example cube`), you can move around smoothly (drag and drop or arrow keys)

If you edit `/examples/cube/scenes.rs` to add `plane_mode: Some(Face::Back),` to `CreatePortal` in `setup_portal_cube_face` ([around line 32](https://github.com/Selene-Amanita/bevy_basic_portals/blob/main/examples/cube/scenes.rs#L32)), this activates the "deactivate/activate optimisation", and moving the cube around isn't smooth anymore.

## Potential Relevant system information

- Rust version:
- Operating System: Linux Mint MATE 1.26, x11
- `AdapterInfo { name: "Intel(R) Xe Graphics (TGL GT2)", vendor: 32902, device: 39497, device_type: IntegratedGpu, driver: "Intel open-source Mesa driver", driver_info: "Mesa 22.2.5", backend: Vulkan }`

Start by comparing the provided minimal example with `activate_and_deactivate_other_cameras` commented out and enabled, using a high `CAMERA_COUNT`; the issue reports the slowdown in release mode too. The relevant entry points in the repro are `activate_and_deactivate_other_cameras` and `setup_other_cameras`; investigate how changing `Camera.is_active` affects rendering. Done means explaining or fixing the reported slowdown so activating and deactivating cameras does not make rendering jerky.

Written by the indexing model from the issue text.