instadeepai / instadeepai/winnow
feat: merged predict table with CSV, TSV or Parquet output
- Dominant language
- Python
- Stars
- 11
- Forks
- 2
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 1
Description
## Summary
`winnow predict` always writes two CSVs (`metadata.csv` and `preds_and_fdr_metrics.csv`). Downstream tools (including INFlow) often want a single PSM table, and parquet/TSV for I/O. Split CSV should remain the default.
## Current behaviour
`_save_predict_outputs` splits columns then writes CSV only. `CalibrationDataset.save_metadata` already knows .csv and .parquet but predict does not use it for TSV or a merged table.
## Proposed config (`predict.yaml`)
Keep `output_folder`. Add two knobs:
```{yaml}
# split (default): metadata + preds_and_fdr_metrics
# merged: one table with metadata, predictions and FDR columns
output_layout: split
# csv (default) | tsv | parquet
output_file_format: csv
```
File names (same stems, suffix from format):
| Layout | Files |
| --- | --- |
| `split` | `metadata.{csv,tsv,parquet}`, `preds_and_fdr_metrics.{csv,tsv,parquet}` |
| `merged` | `calibrated_psms.{csv,tsv,parquet}` |
CSV: comma, header, no index (as today). TSV: tab. Parquet: `format_metadata_for_export()` then `to_parquet` so nested `mz_array` / `intensity_array` stay arrays rather than stringified lists.
## Behaviour
- Default `split` + `csv` is bit-compatible with current outputs (same columns, names and CSV dialect).
- `merged` is the pre-split table: spectrum metadata, features, prediction and FDR columns, still FDR-filtered.
- Invalid `output_layout` / `output_file_format` → clear `ValueError`.
- Nested spectrum arrays: CSV/TSV may stay stringified (current export); parquet should keep native lists.
## Docs/tests
- docs/cli.md prediction output, docs/configuration.md, comments in predict.yaml.
- Unit tests on `_save_predict_outputs` (or a thin writer helper): split vs merged; csv/tsv/parquet; default path still produces the two current CSVs; merged contains `spectrum_id`, `prediction`, confidence, FDR columns and a metadata column such as `precursor_mz`.
### Description & Purpose:
_No response_
### Additional Notes:
_No response_
Contributor guide
Research direction
Start at `_save_predict_outputs` and the `predict.yaml` configuration comments; compare its current split CSV behavior with `CalibrationDataset.save_metadata` and `format_metadata_for_export()`. Add or update unit tests for split and merged layouts across CSV, TSV, and Parquet, then update `docs/cli.md` and `docs/configuration.md`. Done means the default remains bit-compatible and each requested layout and format produces the documented files and columns.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cli, data, documentation, testing
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100