fluent / fluent/fluentd

messagepack bug in http (in) plugin

Open
#4,346 8 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Ruby
Stars
13.6k
Forks
1.4k
Avg merge
1d 3h
Merged PRs (30d)
20

Description

### Describe the bug

The http (in) plugin will not accept a messagepack request that should succeed if it accepts a messagepack request that fails.

### To Reproduce

## Create

```bash
mkdir -p /containers/fluentd
cd /containers/fluentd
mkdir ./output
ls -lnd ./output
```

### compose.yaml

`/containers/fluentd/compose.yaml`

```compose.yaml
services:
fluentd:
restart: always
image: fluent/fluentd:edge
command: fluentd -c /fluentd/etc/fluent.conf
user: 1000:1000
ports:
- 8888:8888
volumes:
- ./fluent.conf:/fluentd/etc/fluent.conf:ro
- ./output:/var/fluentd/output
```

### fluent.conf

`/containers/fluentd/fluent.conf`

```xml
# http input

@type http
port 8888
bind 0.0.0.0

# file output

@type file
path /var/fluentd/output/myapp.access

timekey 86400
timekey_wait 600
timekey_use_utc true

```

### compose up

```bash
docker compose up -d
```

## Check

### 1. Successful Requests

```bash
msgpack=`echo -e "\x81\xa3bar\xa3bar"`

curl \
-X POST \
-H "Content-Type: application/x-www-form-urlencoded" \
-d "msgpack=$msgpack" \
http://localhost:8888/myapp.access
```

### 2. Failure Requests

```bash
msgpack=`echo -e "\x81\xa3bar\xa3bar"`

curl \
-X POST \
-H "Content-Type: application/msgpack" \
-d "msgpack=$msgpack" \
http://localhost:8888/myapp.access
```

```
400 Bad Request
undefined method `delete' for 109:Integer

if t = @default_keep_time_key ? record[@parser_time_key] : record.delete(@parser_time_key)
^^^^^^
```

### 3. Same request as "1."

```bash
msgpack=`echo -e "\x81\xa3bar\xa3bar"`

curl \
-X POST \
-H "Content-Type: application/x-www-form-urlencoded" \
-d "msgpack=$msgpack" \
http://localhost:8888/myapp.access
```

```
400 Bad Request
undefined method `delete' for 115:Integer

if t = @default_keep_time_key ? record[@parser_time_key] : record.delete(@parser_time_key)
^^^^^^^
```

### Expected behavior

3. Same request as "1." succeeds

### Your Environment

```markdown
- Fluentd version: 1.16.2
- TD Agent version:
- Operating system: Ubuntu 22.04.3 LTS
- Kernel version: 5.15.0-87-generic
```

### Your Configuration

```apache
# http input

@type http
port 8888
bind 0.0.0.0

# file output

@type file
path /var/fluentd/output/myapp.access

timekey 86400
timekey_wait 600
timekey_use_utc true

```

### Your Error Log

```shell
fluentd -c /fluentd/etc/fluent.conf
2023-11-20 01:27:06 +0000 [info]: init supervisor logger path=nil rotate_age=nil rotate_size=nil
2023-11-20 01:27:06 +0000 [info]: parsing config file is succeeded path="/fluentd/etc/fluent.conf"
2023-11-20 01:27:06 +0000 [info]: gem 'fluentd' version '1.16.2'
2023-11-20 01:27:06 +0000 [info]: using configuration file:

@type http
port 8888
bind "0.0.0.0"


@type file
path "/var/fluentd/output/myapp.access"

timekey 86400
timekey_wait 600
timekey_use_utc true
path "/var/fluentd/output/myapp.access"

2023-11-20 01:27:06 +0000 [info]: starting fluentd-1.16.2 pid=7 ruby="3.1.4"
2023-11-20 01:27:06 +0000 [info]: spawn command to main: cmdline=["/usr/bin/ruby", "-Eascii-8bit:ascii-8bit", "/usr/bin/fluentd", "-c", "/fluentd/etc/fluent.conf", "--plugin", "/fluentd/plugins", "--under-supervisor"]
2023-11-20 01:27:07 +0000 [info]: #0 init worker0 logger path=nil rotate_age=nil rotate_size=nil
2023-11-20 01:27:07 +0000 [info]: adding match pattern="myapp.access" type="file"
2023-11-20 01:27:07 +0000 [info]: adding source type="http"
2023-11-20 01:27:07 +0000 [info]: #0 starting fluentd worker pid=16 ppid=7 worker=0
2023-11-20 01:27:07 +0000 [info]: #0 fluentd worker is now running worker=0
2023-11-20 01:27:24 +0000 [error]: #0 failed to process request error_class=NoMethodError error="undefined method `delete' for 109:Integer\n\n if t = @default_keep_time_key ? record[@parser_time_key] : record.delete(@parser_time_key)\n ^^^^^^^"
2023-11-20 01:27:53 +0000 [error]: #0 failed to process request error_class=NoMethodError error="undefined method `delete' for 115:Integer\n\n if t = @default_keep_time_key ? record[@parser_time_key] : record.delete(@parser_time_key)\n
```

### Additional context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.