fluent / fluent/fluent-plugin-webhdfs

Can hdfs path use ${tag} or ${record} ? just like: /data/hdfs/${tag}.#{Socket.gethostname}.log

Open
#64 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Ruby
Stars
60
Forks
35
PR merge metrics
No merged PRs in 30d

Description

Hello,
I use 'fluent-plugin-webhdfs' version '1.2.2', fluentd version: 1.4
My log file looks like:
`interface=login&time=20170925&...others fields...`
In my td-agent config file, I named my log as a tag:interface.login.hdfs
```

@type tail
format none
path /data/td-agent/logs/interface.log
pos_file /etc/td-agent/pos/interface.pos
read_from_head true
tag interface.**login**.hdfs

@type webhdfs
host *.*.*.*
port 50070
path /data/gamelogs/hdfs/date=${record['time']}/${tag[2]}.#{Socket.gethostname}.log

@type json

```
and I need to get "login" from tag, and get time from my log record, so if there has any method to get these two values?

In some other plugins, I can use $tag[2] or ${record['interface']}, ${record['time']} to get these values,but in this plugin, it doesn't work.
Thanks...

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.