dandi / dandi/s3-log-extraction

Long-term TODOs

Open
#236 0 comments 0 reactions 0 assignees View on GitHub
future
Dominant language
Python
Stars
2
Forks
4
Avg merge
1d 4h
Merged PRs (30d)
9

Description

- [ ] https://github.com/dandi/s3-log-extraction/blob/main/src/s3_log_extraction/summarize/_generate_all_dataset_totals.py#L22
- `# TODO: record progress over` (add progress tracking around the loop; comment also appears truncated)

- [ ] https://github.com/dandi/s3-log-extraction/blob/main/src/s3_log_extraction/summarize/_generate_all_dataset_totals.py#L25
- `# TODO: use better structure for separating mapped activity from summaries` (reorganize layout so `is_dir()` isn't used as a filter)

- [ ] https://github.com/dandi/s3-log-extraction/blob/main/src/s3_log_extraction/config/_config.py#L17
- `# TODO: add basic schema and validation`

- [ ] https://github.com/dandi/s3-log-extraction/blob/main/src/s3_log_extraction/validate/_http_empty_split_pre_validator.py#L14
- `TODO: should add pre-validator that the 8th element of each space split is always one of the known types.`

Contributor guide

No contributing guide indexed for this repository

Research direction

Read the four linked TODO locations in _generate_all_dataset_totals.py, _config.py, and _http_empty_split_pre_validator.py, then inspect the surrounding configuration and validation code. Treat each checkbox as a separate investigation; done means progress tracking, clearer activity/summary separation, basic config schema validation, and validation of the eighth split element are implemented and covered by the project's existing checks.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-engineering
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.