airbnb / airbnb/streamalert

Improvement: create a new CSV parser or update the current one to support input with a header row

Open
#69 0 comments 0 reactions 0 assignees View on GitHub
improvement log parsers
Dominant language
Python
Stars
2.9k
Forks
324
PR merge metrics
No merged PRs in 30d

Description

The current CSVParser class is not designed to properly handle input if it contains a header row.
We should either subclass this parser or create a new one that can handle parsing of csv input that contains a header row. This parser should probably also try to validate the header to ensure the values match the keys we expect to receive. Python's `csv.DictReader` can likely be used for this, or we may be able to convert the CSVParser to strictly use `csv.DictReader` instead of adding a new class (will need to investigate more...)

Contributor guide

Open the contributing guide

Research direction

Start by locating the CSVParser class and reading how current CSV input is consumed. Compare Python's csv.DictReader with the existing behavior, then define whether to extend or replace the parser and how expected headers are validated; done means header-row input parses correctly and invalid headers are rejected, with focused tests added around those cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.