fluent / fluent/fluentd

Make alternative of FileUtils

Open
#1,285 3 comments 0 reactions 0 assignees View on GitHub
enhancement v0.14
Dominant language
Ruby
Stars
13.6k
Forks
1.4k
Avg merge
1d 3h
Merged PRs (30d)
20

Description

Updated: StringUtil.match_regexp is now easy to replace with `Fluent::Plugin::Base#string_safe_encoding`.
There are still `Fluent::FileUtil.writable?` and `#writable_p?`.

`Fluent::Plugin::StringUtil.match_regexp` (implementation was moved to `Fluent::Compat::StringUtil` now) has some implementation problem:
- it provides encode-safety about regular expression matching, but method name doesn't show it
- it uses `$log`, but should use plugin logger
- scrubbing strings is very universal process: not only for regular expression matching

My idea is to add a method like this:

``` ruby
def ensure_safe_encoding(str)
begin
yield str
rescue ArgumentError => e
raise e unless e.message.include?("invalid byte sequence in".freeze)
log.info "invalid byte sequence is replaced in `#{string}`"
str = str.scrub('?')
retry
end
end
```

We can use this method like below:

``` ruby
m = ensure_safe_encoding(line){|s| s =~ /my pattern/}
# or other methods or ...
chars = ensure_safe_encoding(line){|s| s.split }
```

Blockers for implementation are:
- Naming problem of the method
- Namespace? `lib/fluent`? `lib/fluent/util`? or...
- Should it be a method of `Fluent::Plugin::Base`? plugin helper? (If true, we'll have so many helpers...) or just Mixin? (We're eliminating mixins, but...)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.