fluent / fluent/fluentd

FluentD domain socket is not compliant with Proto v1 - no ack

Open
#1,843 1 comment 0 reactions 0 assignees View on GitHub
v1
Dominant language
Ruby
Stars
13.6k
Forks
1.4k
Avg merge
1d 3h
Merged PRs (30d)
20

Description

Ack is not sent when using domain sockets. Domain sockets are reliable but this does not guarantee at-least-once promise of "chunk" and they should be otherwise treated as TCP as far as reliability goes.

FluentD: v1.1.0
OS: Fedora 27

TCP:
```
Establishing connection to ('127.0.0.1', 24224)
2018-02-03 23:51:09 +0000 [trace]: #0 connected fluent socket addr="172.17.0.1" port=57326
2018-02-03 23:51:09 +0000 [trace]: #0 accepted fluent socket addr="172.17.0.1" port=57326
2018-02-03 23:51:09 +0000 [trace]: #0 [output1] adding metadata instance=47187495938860 metadata=#
2018-02-03 23:51:09 +0000 [trace]: #0 [output1] writing events into buffer instance=47187495938860 metadata_size=1
2018-02-03 23:51:09 +0000 [trace]: #0 sent response to fluent socket address="172.17.0.1" response={"ack"=>"G1zwYgk9EeiRN+SzGGe9Aw=="}
2018-02-03 23:51:09 +0000 [trace]: #0 [output1] adding metadata instance=47187495938860 metadata=#
2018-02-03 23:51:09 +0000 [trace]: #0 [output1] adding metadata instance=47187495938860 metadata=#
2018-02-03 23:51:09 +0000 [trace]: #0 [output1] writing events into buffer instance=47187495938860 metadata_size=1
Sending ['tag-name', 1517701869, {'value-x': 'a', 'value-y': 1}, {'size': 1, 'chunk': b'G1zwYgk9EeiRN+SzGGe9Aw=='}]
Sending ['tag-name', ((1517701869, {'value-x': 'a', 'value-y': 1}), (1517701869, {'value-x': 'm', 'value-b': 200})), {'size': 2, 'chunk': b'G10bggk9EeiRN+SzGGe9Aw=='}]
On , , 6, '', ('127.0.0.1', 24224)), started daemon 140366872520448)> received: {'ack': b'G1zwYgk9EeiRN+SzGGe9Aw=='}
2018-02-03 23:51:09 +0000 [trace]: #0 sent response to fluent socket address="172.17.0.1" response={"ack"=>"G10bggk9EeiRN+SzGGe9Aw=="}
On , , 6, '', ('127.0.0.1', 24224)), started daemon 140366872520448)> received: {'ack': b'G10bggk9EeiRN+SzGGe9Aw=='}
Connection , , 6, '', ('127.0.0.1', 24224)), started daemon 140366872520448)> remote closed while reading
Scheduling close on , , 6, '', ('127.0.0.1', 24224)), started daemon 140366872520448)>
Acknowledging chunk b'G1zwYgk9EeiRN+SzGGe9Aw=='
Acknowledging chunk b'G10bggk9EeiRN+SzGGe9Aw=='
Closing , , 6, '', ('127.0.0.1', 24224)), started daemon 140366872520448)>
```

UNIX:
```
Establishing connection to /home//fluent-logger-python/tests/fluent_sock/fluent.sock
Sending ['tag-name', 1517701872, {'value-x': 'a', 'value-y': 1}, {'size': 1, 'chunk': b'HSeVlgk9EeiRN+SzGGe9Aw=='}]
Sending ['tag-name', ((1517701872, {'value-x': 'a', 'value-y': 1}), (1517701872, {'value-x': 'm', 'value-b': 200})), {'size': 2, 'chunk': b'HSev/gk9EeiRN+SzGGe9Aw=='}]
2018-02-03 23:51:12 +0000 [trace]: #0 accepted fluent socket from ':': object_id=47187497540320
2018-02-03 23:51:12 +0000 [trace]: #0 [output1] adding metadata instance=47187495938860 metadata=#
2018-02-03 23:51:12 +0000 [trace]: #0 [output1] writing events into buffer instance=47187495938860 metadata_size=1
2018-02-03 23:51:12 +0000 [trace]: #0 [output1] adding metadata instance=47187495938860 metadata=#
2018-02-03 23:51:12 +0000 [trace]: #0 [output1] adding metadata instance=47187495938860 metadata=#
2018-02-03 23:51:12 +0000 [trace]: #0 [output1] writing events into buffer instance=47187495938860 metadata_size=1
2018-02-03 23:51:12 +0000 [trace]: #0 [output1] adding metadata instance=47187495938860 metadata=#
2018-02-03 23:51:12 +0000 [trace]: #0 [output1] writing events into buffer instance=47187495938860 metadata_size=1
Closing , , 6, '', '/home//fluent-logger-python/tests/fluent_sock/fluent.sock'), started daemon 140366872520448)>
2018-02-03 23:51:15 +0000 [trace]: #0 closed fluent socket object_id=47187497540320
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.