ailia-ai / ailia-ai/ailia-sdk-cpp

ailiaOpenStreamFileW / ailiaUpdate: Infinite loop (CPU hang) on malformed Caffe prototxt — two separate DoS vectors

Đang mở
#8 0 bình luận 0 reaction 0 người được giao Xem trên GitHub
Ngôn ngữ chính
C
Star
1
Fork
1
Chỉ số merge pull request
Không có pull request nào được merge trong 30 ngày

Mô tả

## Summary

Two denial-of-service vulnerabilities were found in `ailia.dll` that cause the calling thread to spin indefinitely when processing a malformed Caffe-format `.prototxt` file via the public API. Both issues were confirmed live against the Windows x64 release of the ailia SDK.

---

### Issue 1 — Parser infinite loop in `ailiaOpenStreamFileW`

**Affected API:** `ailiaOpenStreamFileW` (and `ailiaOpenStreamEx`)

**Root cause:**
Inside the Caffe prototxt text parser, the stream-state check after reading a field value uses the mask `(state & 6)` — which tests only `failbit (0x2)` and `badbit (0x4)`. When the file ends mid-value (e.g. `bottom:` with no value before EOF), `eofbit (0x1)` is set but the outer loop condition `(state & 6) == 0` remains true, so the loop never exits.

**Minimal reproducer (18 bytes):**

```
name:"tiny"
layer {
bottom:
```

Pass this as the prototxt path to `ailiaOpenStreamFileW`. The calling thread immediately enters a busy-loop consuming 100% of one CPU core and never returns.

**Call stack (observed, ailia.dll loaded at `0x00007FFE29EE0000`):**

```
ailia!ailiaFormatConvert+0x2aec01 ← infinite loop body
ailia!ailiaFormatConvert+0x2ade75
ailia!ailiaOpenStreamEx+0x726
ailia!ailiaOpenStreamFileW+0x145 ← public API entry point
```

**Decompiled loop condition (Ghidra, `FUN_1802bf670`):**

```c
while (uVar4 != 0xffffffff) { // snextc() loops until EOF
/* append char to buffer */
uVar4 = snextc(streambuf);
}
uVar4 = 1; // set eofbit only
if (bVar2) goto skip_failbit; // char was read → skip setting failbit!
uVar4 |= 2; // failbit — never reached when bVar2=true
setstate(uVar4);
// outer loop checks (state & 6) == 0 → true even with eofbit set → loops forever
```

---

### Issue 2 — Integer overflow in `ailiaUpdate` causes infinite loop

**Affected API:** `ailiaUpdate`

**Root cause:**
The convolution output-size formula uses a 32-bit signed intermediate. For a Caffe prototxt with `pad_h` / `pad_w` set to values near `(INT32_MAX − 10) / 2`, the expression `2 × pad_h + 10` overflows signed int32, producing a negative `output_h`. A downstream loop then spins on the negative size indefinitely.

**Minimal reproducer:**
Take any valid Caffe prototxt that ailia accepts, replace every `pad_h` and `pad_w` value with `1073741820`, load it with `ailiaOpenStreamFileW` + `ailiaOpenWeightFileW` (any matching caffemodel), then call `ailiaUpdate`. The function never returns.

```
pad_h: 1073741820
pad_w: 1073741820
```

**Arithmetic:**

```
output_h = (input_h + 2 × pad_h − kernel_h) / stride_h + 1
= (16 + 2 × 1073741820 − 7) / 1 + 1
= 2147483650 ← overflows int32
= −2147483646 ← negative → infinite loop
```

**Boundary observations** (kernel_h=7, input=16×16):

| pad_h | Behaviour |
|-----------------|------------------------------------------------------|
| ≤ 896 | `ailiaUpdate` returns 0 (success) |
| 960 – 65534 | Returns −5 or −128 — graceful error |
| **1073741820** | **`ailiaUpdate` never returns — CPU hang** |
| 2147483648 | Returns 0 with degenerate 0×0 output (harmless) |

---

### Environment

| Item | Value |
|---------------|----------------------------------------------------|
| OS | Windows 11 Pro 10.0.26200 (x64) |
| ailia.dll | version as shipped with CLIP STUDIO PAINT 1.5 |
| Architecture | x64 |
| API language | C / Python ctypes |

---

### Impact

Both bugs allow a crafted `.prototxt` file to cause 100% CPU utilisation on one core with no timeout or error recovery in the calling process. Any application that passes user-controlled or externally-sourced prototxt files to `ailiaOpenStreamFileW` / `ailiaUpdate` is affected.

**Suggested fix:**
- Issue 1: After reading a field value, check `eofbit` in addition to `failbit`/`badbit`, or call `good()` / `fail()` to test the full stream state.
- Issue 2: Widen the `output_h` intermediate to `int64_t` (or `size_t`) before computing the size, and add a negative-result guard before allocation.

Hướng dẫn đóng góp

Chưa lập chỉ mục được hướng dẫn đóng góp cho kho mã nguồn này

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.