embulk / embulk/embulk-standards

guess fails when header line contains NULL

Open
#14 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
0
Forks
0
PR merge metrics
No merged PRs in 30d

Description

I assume the problem is below.

- determine header line logic for the case where contain null.
- skip_heaer_line is not passed to guesser

Question:
After this is issued is fixed, could you publish a new version of embulk 0.9.*
embulk/embulk#1240 is in progress for 0.10 branch. Should I make PR to https://github.com/embulk/embulk-util-guess?

Issue Type: Bug Report
-----------------------

* Write the following environmental information.
* OS version MacOS 11
* Java version
* Embulk version 0.10.23 or 9.23
* Your Embulk configuration (YAML)
* Plugin versions
* Write all what you did, e.g. your commands executed
Please see below

* Write what you expected and observed actually
expected:
Guess is config without errors.

observed:
I attached error log bewlow.

config

```
in:
type: file
path_prefix: ./data.part_00000
skip_header_lines: 1
null_string: ""
parser:
type: csv
out:
type: stdout
```

data.part_00000

```
"-1" "hogege" "-1" "" \N "" "" "" "" "" "" "" "1" "2020-06-27 00:00:00" "" "2020-12-17 00:00:00 UTC" "gegege" "" ""
"8" "" "8" "" "1" "" "" "" "" "" "" "" "0" "" "" "2020-12-12 00:00:00 UTC" "foo" "" ""
"8" "" "8" "" "1" "" "" "" "" "" "" "" "0" "" "" "2020-12-12 00:00:00 UTC" "foo" "" ""
"8" "" "8" "" "1" "" "" "" "" "" "" "" "0" "" "" "2020-12-12 00:00:00 UTC" "foo" "" ""
"8" "" "8" "" "1" "" "" "" "" "" "" "" "0" "" "" "2020-12-12 00:00:00 UTC" "foo" "" ""
"8" "" "8" "" "1" "" "" "" "" "" "" "" "0" "" "" "2020-12-12 00:00:00 UTC" "foo" "" ""

```

log

```
> java -jar embulk-0.10.23.jar -X jruby=file:/Users/ken/ws/pn/embulk10/jruby-complete-9.1.15.0.jar guess config.yml 7.6s < Fri Dec 18 12:47:00 2020
2020-12-18 12:47:03.106 +0900 [INFO] (main): Started Embulk v0.10.23
2020-12-18 12:47:03.200 +0900 [INFO] (0001:guess): Listing local files at directory '.' filtering filename by prefix 'preview.part_00000'
2020-12-18 12:47:03.201 +0900 [INFO] (0001:guess): "follow_symlinks" is set false. Note that symbolic links to directories are skipped.
2020-12-18 12:47:03.207 +0900 [INFO] (0001:guess): Loading files [./preview.part_00000.txt]
2020-12-18 12:47:03.216 +0900 [INFO] (0001:guess): Try to read 32,768 bytes from input source
2020-12-18 12:47:05.404 +0900 [INFO] (0001:guess): Gem's home and path are set by system configs "gem_home": "/Users/ken/.embulk/lib/gems", "gem_path": ""
2020-12-18 12:47:06.184 +0900 [INFO] (0001:guess): Loaded plugin embulk (0.10.23)
2020-12-18 12:47:06.197 +0900 [INFO] (0001:guess): Loaded plugin embulk (0.10.23)
2020-12-18 12:47:06.211 +0900 [INFO] (0001:guess): Loaded plugin embulk (0.10.23)
2020-12-18 12:47:06.217 +0900 [INFO] (0001:guess): Loaded plugin embulk (0.10.23)
org.jruby.exceptions.RaiseException: (NoMethodError) undefined method `strip' for nil:NilClass
at org.jruby.RubyArray.map(org/jruby/RubyArray.java:2487)
at RUBY.guess_lines(uri:classloader:/gems/embulk-0.10.23-java/lib/embulk/guess/csv.rb:165)
at RUBY.guess(uri:classloader:/gems/embulk-0.10.23-java/lib/embulk/guess_plugin.rb:121)
at RUBY.guess(uri:classloader:/gems/embulk-0.10.23-java/lib/embulk/guess_plugin.rb:24)
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.