dmlc / dmlc/MXNet.jl

CSVIter crashes claiming different row length

Open
#214 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
No language data
Stars
369
Forks
63
PR merge metrics
No merged PRs in 30d

Description

Hi.

I'm trying to use CSVIter to load a dataset I have in CSV format. The data file has 25 columns (some of the can be empty, especially the last columns).

While executing, it outputs the following error trace:

> julia> using MXNet;
>
> julia> train_provider = mx.CSVIter(data_csv = "/home/laketide/work/Alejandro/output_training_data.csv", data_shape =(1,25), label_csv = "/home/laketide/work/Alejandro/output_training_labels.csv", label_shape = (1,2), batch_size = 1)
> [15:45:59] /home/laketide/mxnet/dmlc-core/include/dmlc/./logging.h:300: [15:45:59] src/io/iter_csv.cc:105: Check failed: row.length == shape.Size() (24 vs. 25) The data size in CSV do not match size of shape: specified shape=(25,1), the csv row-length=24
>
> Stack trace returned 9 entries:
> [bt] (0) /home/laketide/mxnet/lib/libmxnet.so(_ZN4dmlc15LogMessageFatalD1Ev+0x3c) [0x7f5f4df6bfac]
> [bt] (1) /home/laketide/mxnet/lib/libmxnet.so(_ZN5mxnet2io7CSVIter7AsTBlobERKN4dmlc3RowIjEERKN4nnvm6TShapeE+0x3aa) [0x7f5f4e73ae2a]
> [bt] (2) /home/laketide/mxnet/lib/libmxnet.so(_ZN5mxnet2io7CSVIter4NextEv+0x19c) [0x7f5f4e73e5ac]
> [bt] (3) /home/laketide/mxnet/lib/libmxnet.so(_ZN5mxnet2io11BatchLoader4NextEv+0x87) [0x7f5f4e74d617]
> [bt] (4) /home/laketide/mxnet/lib/libmxnet.so(_ZZN5mxnet2io14PrefetcherIter4InitERKSt6vectorISt4pairINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES9_ESaISA_EEENKUlPPNS_9DataBatchEE_clESH_+0x39) [0x7f5f4e7406d9]
> [bt] (5) /home/laketide/mxnet/lib/libmxnet.so(_ZZN4dmlc12ThreadedIterIN5mxnet9DataBatchEE4InitESt8functionIFbPPS2_EES4_IFvvEEENKUlvE_clEv+0x1ff) [0x7f5f4e746d0f]
> [bt] (6) /usr/bin/../lib/x86_64-linux-gnu/libstdc++.so.6(+0xb8c80) [0x7f61912c9c80]
> [bt] (7) /lib/x86_64-linux-gnu/libpthread.so.0(+0x76ba) [0x7f619304e6ba]
> [bt] (8) /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f6192d8482d]
>
> terminate called after throwing an instance of 'dmlc::Error'
> what(): [15:45:59] src/io/iter_csv.cc:105: Check failed: row.length == shape.Size() (24 vs. 25) The data size in CSV do not match size of shape: specified shape=(25,1), the csv row-length=24
>
> Stack trace returned 9 entries:
> [bt] (0) /home/laketide/mxnet/lib/libmxnet.so(_ZN4dmlc15LogMessageFatalD1Ev+0x3c) [0x7f5f4df6bfac]
> [bt] (1) /home/laketide/mxnet/lib/libmxnet.so(_ZN5mxnet2io7CSVIter7AsTBlobERKN4dmlc3RowIjEERKN4nnvm6TShapeE+0x3aa) [0x7f5f4e73ae2a]
> [bt] (2) /home/laketide/mxnet/lib/libmxnet.so(_ZN5mxnet2io7CSVIter4NextEv+0x19c) [0x7f5f4e73e5ac]
> [bt] (3) /home/laketide/mxnet/lib/libmxnet.so(_ZN5mxnet2io11BatchLoader4NextEv+0x87) [0x7f5f4e74d617]
> [bt] (4) /home/laketide/mxnet/lib/libmxnet.so(_ZZN5mxnet2io14PrefetcherIter4InitERKSt6vectorISt4pairINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES9_ESaISA_EEENKUlPPNS_9DataBatchEE_clESH_+0x39) [0x7f5f4e7406d9]
> [bt] (5) /home/laketide/mxnet/lib/libmxnet.so(_ZZN4dmlc12ThreadedIterIN5mxnet9DataBatchEE4InitESt8functionIFbPPS2_EES4_IFvvEEENKUlvE_clEv+0x1ff) [0x7f5f4e746d0f]
> [bt] (6) /usr/bin/../lib/x86_64-linux-gnu/libstdc++.so.6(+0xb8c80) [0x7f61912c9c80]
> [bt] (7) /lib/x86_64-linux-gnu/libpthread.so.0(+0x76ba) [0x7f619304e6ba]
> [bt] (8) /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f6192d8482d]
>
>
> signal (6): Aborted
> while loading no file, in expression starting on line 0
> Allocations: 3195248 (Pool: 3193533; Big: 1715); GC: 3
> Aborted (core dumped)
>

I've checked all the lines. All of them have 24 commas, so it has 25 columns.

Thanks in advance.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.