MISP / MISP/misp-modules

CSV Import - 'NoneType' object is not iterable

Open
#179 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
377
Forks
280
Avg merge
2d 2h
Merged PRs (30d)
10

Description

When trying to import a CSV I get the following error:

Apr 19 15:08:34 dsgnmisp01 misp-modules: 2018-04-19 15:08:34,735 - misp-modules - ERROR - Something went wrong:
Apr 19 15:08:34 dsgnmisp01 misp-modules: Traceback (most recent call last):
Apr 19 15:08:34 dsgnmisp01 misp-modules: File "/usr/lib/python3.4/site-packages/misp_modules/__init__.py", line 197, in post
Apr 19 15:08:34 dsgnmisp01 misp-modules: response = yield tornado.gen.with_timeout(timeout, self.run_request(jsonpayload))
Apr 19 15:08:34 dsgnmisp01 misp-modules: File "/usr/lib64/python3.4/site-packages/tornado/gen.py", line 1099, in run
Apr 19 15:08:34 dsgnmisp01 misp-modules: value = future.result()
Apr 19 15:08:34 dsgnmisp01 misp-modules: File "/usr/lib64/python3.4/asyncio/futures.py", line 274, in result
Apr 19 15:08:34 dsgnmisp01 misp-modules: raise self._exception
Apr 19 15:08:34 dsgnmisp01 misp-modules: File "/usr/lib64/python3.4/concurrent/futures/thread.py", line 54, in run
Apr 19 15:08:34 dsgnmisp01 misp-modules: result = self.fn(*self.args, **self.kwargs)
Apr 19 15:08:34 dsgnmisp01 misp-modules: File "/usr/lib/python3.4/site-packages/misp_modules/__init__.py", line 185, in run_request
Apr 19 15:08:34 dsgnmisp01 misp-modules: response = mhandlers[x['module']].handler(q=jsonpayload)
Apr 19 15:08:34 dsgnmisp01 misp-modules: File "modules/import_mod/csvimport.py", line 117, in handler
Apr 19 15:08:34 dsgnmisp01 misp-modules: csv_parser.parse_data(data.split('\n'))
Apr 19 15:08:34 dsgnmisp01 misp-modules: File "modules/import_mod/csvimport.py", line 28, in parse_data
Apr 19 15:08:34 dsgnmisp01 misp-modules: self.delimiter, self.length = self.findDelimiter()
Apr 19 15:08:34 dsgnmisp01 misp-modules: TypeError: 'NoneType' object is not iterable
Apr 19 15:08:34 dsgnmisp01 misp-modules: ERROR:misp-modules:Something went wrong:
Apr 19 15:08:34 dsgnmisp01 misp-modules: Traceback (most recent call last):
Apr 19 15:08:34 dsgnmisp01 misp-modules: File "/usr/lib/python3.4/site-packages/misp_modules/__init__.py", line 197, in post
Apr 19 15:08:34 dsgnmisp01 misp-modules: response = yield tornado.gen.with_timeout(timeout, self.run_request(jsonpayload))
Apr 19 15:08:34 dsgnmisp01 misp-modules: File "/usr/lib64/python3.4/site-packages/tornado/gen.py", line 1099, in run
Apr 19 15:08:34 dsgnmisp01 misp-modules: value = future.result()
Apr 19 15:08:34 dsgnmisp01 misp-modules: File "/usr/lib64/python3.4/asyncio/futures.py", line 274, in result
Apr 19 15:08:34 dsgnmisp01 misp-modules: raise self._exception
Apr 19 15:08:34 dsgnmisp01 misp-modules: File "/usr/lib64/python3.4/concurrent/futures/thread.py", line 54, in run
Apr 19 15:08:34 dsgnmisp01 misp-modules: result = self.fn(*self.args, **self.kwargs)
Apr 19 15:08:34 dsgnmisp01 misp-modules: File "/usr/lib/python3.4/site-packages/misp_modules/__init__.py", line 185, in run_request
Apr 19 15:08:34 dsgnmisp01 misp-modules: response = mhandlers[x['module']].handler(q=jsonpayload)
Apr 19 15:08:34 dsgnmisp01 misp-modules: File "modules/import_mod/csvimport.py", line 117, in handler
Apr 19 15:08:34 dsgnmisp01 misp-modules: csv_parser.parse_data(data.split('\n'))
Apr 19 15:08:34 dsgnmisp01 misp-modules: File "modules/import_mod/csvimport.py", line 28, in parse_data
Apr 19 15:08:34 dsgnmisp01 misp-modules: self.delimiter, self.length = self.findDelimiter()
Apr 19 15:08:34 dsgnmisp01 misp-modules: TypeError: 'NoneType' object is not iterable

It seems something inside findDelimiter() is causing the issue.

Does someone have a working test.csv (with corresponding headers) I could troubleshoot with?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in modules/import_mod/csvimport.py, especially handler() and findDelimiter(), using the traceback as the entry point. Reproduce the failure with a CSV and its headers, then determine the expected delimiter and header handling; done means a valid CSV import no longer raises the reported NoneType iteration error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.