seeed_xiao_esp32c3 Board Unavailable until connecting to port

### The problem

After uploading new firmware to my seeed_xiao_esp32c3 board, I noticed that is stays offline until I connect to its USB port and open the logs in es-home through USB. If I don't connect to the Logs it doesn't ever get online but also doesn't use the fallback hotspot...

### Which version of ESPHome has the issue?

2023.8.3

### What type of installation are you using?

Home Assistant Add-on

### Which version of Home Assistant has the issue?

2023.8.1

### What platform are you using?

ESP32

### Board

ESP32-c3

### Component causing the issue

_No response_

### Example YAML snippet

```yaml
esphome:
name: meter-water
friendly_name: meter-water
includes:
- my_custom_component.h

esp32:
board: seeed_xiao_esp32c3
framework:
type: arduino

# Enable logging
logger:

# Enable Home Assistant API
api:
encryption:
key: "Fs2wo4KFz8RZtfULDX/hGr3W2sA/vASOqBuHWl+uDss="

ota:
password: "ef7472a1da570a3be2f215c9dbafa795"

wifi:
ssid: !secret wifi_ssid
password: !secret wifi_password

web_server:
port: 80

sensor:
- platform: custom
lambda: |-
auto my_custom = new MyCustomSensor();
App.register_component(my_custom);
return {my_custom->water_liter_sensor, my_custom->sensa, my_custom->sensb, my_custom->sensc};

sensors:
- name: "water_liter_sensor"
unit_of_measurement: L
state_class: total_increasing
device_class: water
accuracy_decimals: 2
- name: "sensa"
accuracy_decimals: 2
- name: "sensb"
accuracy_decimals: 2
- name: "sensc"
accuracy_decimals: 2
```

### Anything in the logs that might be useful for us?

After opening the logs, the first thing showing up is this:

```txt
There have been 3 suspected unsuccessful boot attempts.

```

### Additional information

_No response_