external_components not properly fetching (before yaml check) / sometimes not working at all

### The problem

The external_components function is a bit flaky for me, when overwriting an already existing component with an updated one for testing.

## Issues I've noticed
- Using just a branch, will lead to no updates being done, after pushing new commits to Github to the branch.
- Using tags will sometimes not work at all, without any notification about not using the external_component. This will lead to complains by the yaml checker if there were changes to the syntax as the only way to find out that something was going wrong. If there were no changes to the syntax, there's no way of knowing that this happened.
- After pushing a new tag, even when waiting two minutes, it will still complain that this path does not exist. A retry will immediately fetch the new tag and build (in most cases).

### Observations
- Sometimes it helps to wait just half an hour, sometimes it won't.
- Cleaning the build files don't help
- Pushing the same commit to a new tag will help sometimes.

### Which version of ESPHome has the issue?

2024.9.2 / 2024.9.1

### What type of installation are you using?

Home Assistant Add-on

### Which version of Home Assistant has the issue?

2024.10.0b3 / 2024.9.0

### What platform are you using?

ESP8266

### Board

Wemos D1 mini

### Component causing the issue

external_components

### Example YAML snippet

```yaml
external_components:
- source: github://RubenKelevra/esphome_esphome@tcs34725_spec_based_irradiance_output_v0.23
components: [ tcs34725 ]
```

### Anything in the logs that might be useful for us?

```txt
#### Not fetching the update at all:

INFO ESPHome 2024.9.2
INFO Reading configuration /config/esphome/ambient-light-sensor.yaml...
INFO Detected timezone 'Europe/Berlin'
Failed config

sensor.tcs34725: [source /config/esphome/ambient-light-sensor.yaml:88]
platform: tcs34725
i2c_id: bus_a

[red_channel_irradiance] is an invalid option for [sensor.tcs34725]. Did you mean [red_channel]?
red_channel_irradiance:
name: Ambient Light Sensor Red Channel Irradiance
accuracy_decimals: 1
filters:
- or:
- throttle: 300s
- delta: 0.01

[green_channel_irradiance] is an invalid option for [sensor.tcs34725]. Did you mean [green_channel]?
green_channel_irradiance:
name: Ambient Light Sensor Green Channel Irradiance
accuracy_decimals: 1
filters:
- or:
- throttle: 300s
- delta: 0.01

[blue_channel_irradiance] is an invalid option for [sensor.tcs34725]. Did you mean [blue_channel]?
blue_channel_irradiance:
name: Ambient Light Sensor Blue Channel Irradiance
accuracy_decimals: 1
filters:
- or:
- throttle: 300s
- delta: 0.01

[sensor_saturation] is an invalid option for [sensor.tcs34725]. Please check the indentation.
sensor_saturation:
name: Ambient Light Sensor Sensor Saturation
accuracy_decimals: 1

#### First try of fetching a newly published tag:

INFO ESPHome 2024.9.2
INFO Reading configuration /config/esphome/ambient-light-sensor.yaml...
Failed config

external_components: [source /config/esphome/ambient-light-sensor.yaml:2]
-
Could not find components folder for source. Please check the source contains a 'components' or 'esphome/components' folder.
source: |-
github://RubenKelevra/esphome_esphome@tcs34725_spec_based_irradiance_output_v0.24
components:
- tcs34725

### Hitting retry won't work:

INFO ESPHome 2024.9.2
INFO Reading configuration /config/esphome/ambient-light-sensor.yaml...
INFO Detected timezone 'Europe/Berlin'
Failed config

sensor.tcs34725: [source /config/esphome/ambient-light-sensor.yaml:88]
platform: tcs34725
i2c_id: bus_a

[red_channel_irradiance] is an invalid option for [sensor.tcs34725]. Did you mean [red_channel]?
red_channel_irradiance:
name: Ambient Light Sensor Red Channel Irradiance
accuracy_decimals: 1
filters:
- or:
- throttle: 300s
- delta: 0.01

[green_channel_irradiance] is an invalid option for [sensor.tcs34725]. Did you mean [green_channel]?
green_channel_irradiance:
name: Ambient Light Sensor Green Channel Irradiance
accuracy_decimals: 1
filters:
- or:
- throttle: 300s
- delta: 0.01

[blue_channel_irradiance] is an invalid option for [sensor.tcs34725]. Did you mean [blue_channel]?
blue_channel_irradiance:
name: Ambient Light Sensor Blue Channel Irradiance
accuracy_decimals: 1
filters:
- or:
- throttle: 300s
- delta: 0.01

[sensor_saturation] is an invalid option for [sensor.tcs34725]. Please check the indentation.
sensor_saturation:
name: Ambient Light Sensor Sensor Saturation
accuracy_decimals: 1

#### Changing to v0.23, which is the same commit, does work:

INFO ESPHome 2024.9.2
INFO Reading configuration /config/esphome/ambient-light-sensor.yaml...
INFO Detected timezone 'Europe/Berlin'
INFO Generating C++ source...
INFO Compiling app...
Processing ambient-light-sensor (board: d1_mini; framework: arduino; platform: platformio/espressif8266@4.2.1)
--------------------------------------------------------------------------------
HARDWARE: ESP8266 80MHz, 80KB RAM, 4MB Flash
Dependency Graph
|-- ESP8266WiFi @ 1.0
|-- ESP8266mDNS @ 1.2
|-- noise-c @ 0.1.6
|-- Wire @ 1.0
```

### Additional information

_No response_