out_s3: process crashes (SIGSEGV/SIGABRT) after retry-exhausted chunk is moved to quarantine (v5.1.1)

## Bug Report

**Describe the bug**

`out_s3` crashes the whole Fluent Bit process (alternating `SIGSEGV` and `SIGABRT`) immediately after a chunk exhausts its retries and is moved to quarantine. The crash reproduces deterministically whenever PutObject fails persistently (in our case a 403 `AccessDenied` from a mis-scoped bucket policy). With `Restart=always` under systemd this becomes a crash loop: on each restart the plugin reloads the locally buffered data from `store_dir`, fails again, exhausts retries, quarantines, and crashes again. We saw 26 crashes in ~9 minutes on one host (9 × `SIGSEGV`, 17 × `SIGABRT`), 92 `retry-exhausted chunk moved to quarantine` events and 5 `could not quarantine, deleting retry-exhausted chunk` events in that window.

The trigger is the *retry-exhaustion / quarantine* path, not the 403 itself. Earlier in the same run, dozens of 403s were handled and retried cleanly. The process only dies once a chunk hits `failed to send 6/5 times, will not retry`.

Note: `retry_limit False` is capped to 5 by this version (`[ warn] retry_limit set to unlimited, capping to 5`), so the default "unlimited" configuration also reaches this path after 5 failures.

**To Reproduce**

1. Configure `out_s3` against a bucket the instance is **not** permitted to write to (any persistent 4xx from PutObject will do). We hit it with an instance-profile role and a bucket policy that denied `s3:PutObject`.
2. `retry_limit 5` (or leave it unset / `False`; it is capped to 5).
3. `upload_timeout 1m` to make it fast. Any input that produces records (we used `in_systemd` and `in_tail`).
4. Start Fluent Bit. After the first chunk fails 6 times:

```
[ warn] [output:s3:s3.1] Chunk for tag journal failed to send 6/5 times, will not retry
[ warn] [output:s3:s3.1] retry-exhausted chunk moved to quarantine
[engine] caught signal (SIGSEGV)
```

or, about two thirds of the time, a `SIGABRT` (glibc abort; looks like a double free / heap corruption) with the same preceding lines.

Fuller excerpt (hostnames, bucket names, account and instance IDs redacted):

```
[2026/09/04 14:16:35.658] [ warn] [engine] failed to flush chunk '4061136-1788545794.622011606.flb', retry in 7 seconds: task_id=3, input=systemd.0 > output=s3.1 (out_id=1)
[2026/09/04 14:16:36.614] [ info] [output:s3:s3.1] Running upload timer callback (cb_s3_upload)..
[2026/09/04 14:16:36.614] [ info] [output:s3:s3.1] Sending locally buffered data from previous executions to S3; buffer=/var/lib/fluent-bit/s3-buffer-ops/example-ops-logs
[2026/09/04 14:16:36.614] [ warn] [output:s3:s3.1] Chunk for tag journal failed to send 6/5 times, will not retry
[2026/09/04 14:16:36.614] [ info] [output:s3:s3.1] Sending locally buffered data from previous executions to S3; buffer=/var/lib/fluent-bit/s3-buffer-ops/example-ops-logs
[2026/09/04 14:16:36.614] [ warn] [output:s3:s3.1] Chunk for tag journal failed to send 6/5 times, will not retry
[2026/09/04 14:16:36.614] [ info] [output:s3:s3.0] Sending locally buffered data from previous executions to S3; buffer=/var/lib/fluent-bit/s3-buffer-oth/example-other-logs
[2026/09/04 14:16:36.614] [ warn] [output:s3:s3.1] retry-exhausted chunk moved to quarantine
[2026/09/04 14:16:36.615] [ info] [output:s3:s3.0] Pre-compression chunk size is 65610, After compression, chunk is 2728 bytes
[2026/09/04 14:16:36] [engine] caught signal (SIGSEGV)
[2026/09/04 14:16:36.616] [ info] [output:s3:s3.1] Pre-compression chunk size is 130965, After compression, chunk is 4231 bytes
[2026/09/04 14:16:36.651] [error] [output:s3:s3.0] PutObject API responded with error='AccessDenied', message='User: arn:aws:sts::111122223333:assumed-role/example-instance-role/i-0EXAMPLE0000000000 is not authorized to perform: s3:PutObject on resource: "arn:aws:s3:::example-other-logs/logs/2026/09/04/18/host01.example.internal-journal-20260904-181627-U3ZJi8Eo.log.gz" because no identity-based policy allows the s3:PutObject action'
[2026/09/04 14:16:36.651] [error] [output:s3:s3.0] Raw PutObject response: HTTP/1.1 403 Forbidden
...
[2026/09/04 14:16:36.657] [ warn] [engine] failed to flush chunk '4061136-1788545785.948091653.flb', retry in 19 seconds: task_id=6, input=systemd.0 > output=s3.1 (out_id=1)
#0 0x71c7bf in ???() at ???:0
#1 0x712d11 in ???() at ???:0
#2 0x71653d in ???() at ???:0
#3 0x71a962 in ???() at ???:0
#4 0xcf37a6 in ???() at ???:0
#5 0xffffffffffffffff in ???() at ???:0
fluent-bit.service: Main process exited, code=dumped, status=6/ABRT
fluent-bit.service: Failed with result 'core-dump'.
fluent-bit.service: Scheduled restart job, restart counter is at 1.
```

Note the interleaving: two `out_s3` instances (`s3.0`, `s3.1`) with 4 workers each are active when it happens. The `SIGSEGV` line is emitted between two `Pre-compression chunk size` lines from different output instances, so a race between worker threads on the quarantine / chunk-free path seems plausible.

Once the bucket policy was corrected the same configuration ran without incident and drained the buffered data.

**Expected behavior**

A chunk that exhausts its retries should be quarantined (or dropped) and the process should keep running. A persistent 403 from S3 should never take down the agent, and certainly not in a way that crash-loops and (via `could not quarantine, deleting retry-exhausted chunk`) loses data on every iteration.

**Your Environment**

* Version used: Fluent Bit v5.1.1 (official RPM from `packages.fluentbit.io`)
* Configuration (classic format, sanitized):

```ini
[SERVICE]
flush 1s
daemon Off
log_level info
parsers_file parsers.conf
parsers_file bq-parsers.conf
storage.path /var/lib/fluent-bit/storage
storage.sync normal
storage.checksum off
storage.max_chunks_up 32
storage.backlog.mem_limit 16M
storage.metrics On

[INPUT]
Name systemd
Tag journal
DB /var/lib/fluent-bit/journal.db
Read_From_Tail On
Strip_Underscores On
Lowercase On
storage.type filesystem

[INPUT]
Name tail
Tag app
Path /opt/app/log/*log
DB /var/lib/fluent-bit/app.db
Threaded true
Read_From_Head Off
Skip_Long_Lines On
storage.type filesystem

[OUTPUT]
Name s3
Match *
workers 4
region us-east-2
bucket example-other-logs
total_file_size 1G
upload_timeout 1m
use_put_object On
send_content_md5 On
store_dir /var/lib/fluent-bit/s3-buffer-oth
store_dir_limit_size 2G
s3_key_format /logs/%Y/%m/%d/%H/${HOSTNAME}-$TAG-%Y%m%d-%H%M%S-$UUID.log.gz
compression gzip
retry_limit 5

[OUTPUT]
Name s3
Match *
workers 4
region us-east-2
bucket example-ops-logs
total_file_size 1G
upload_timeout 1m
use_put_object On
send_content_md5 On
store_dir /var/lib/fluent-bit/s3-buffer-ops
store_dir_limit_size 2G
s3_key_format /logs/$TAG/%Y/%m/%d/%H/${HOSTNAME}-%Y%m%d-%H%M%S-$UUID.log.gz
compression gzip
retry_limit 5
```

* Environment name and version: bare EC2 (no containers, no Kubernetes)
* Server type and version: AWS EC2, x86_64, credentials via instance profile (IMDS)
* Operating System and version: Rocky Linux 9.x, kernel 5.14.0-687.x; systemd unit with `Restart=always`, `RestartSec=10`
* Filters and plugins: `in_systemd`, `in_tail`, `out_s3`. No filters.

**Additional context**

* The backtrace printed by the crash handler has no symbols (`???()`). We have ~26 core dumps in `coredumpctl` from this incident and can provide a `gdb` backtrace with debuginfo if you point me at the right package / build.
* `retry_limit False` is capped to 5 in this version, so the default configuration reaches the quarantine path after 5 failures.
* Workaround in the meantime: a much higher `retry_limit`, so exhaustion requires a prolonged outage rather than a brief one.

Start with the out_s3 retry-exhaustion and quarantine path, reproducing the failure using the supplied two-output configuration, persistent S3 403, and retry_limit 5. Inspect the available core dumps with debuginfo and gdb, focusing on the concurrent workers and chunk cleanup; done means exhausted chunks are quarantined or dropped without SIGSEGV/SIGABRT and the Fluent Bit process remains running.

Written by the indexing model from the issue text.