stan-dev / stan-dev/stan

FR: 0_append for csvs

Open
#3,044 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

i/o
Dominant language
C++
Stars
2.8k
Forks
388
Avg merge
2d 17h
Merged PRs (30d)
15

Description

Excuse my general C naivete but would it be possible at all to have stan write it's csvs using O_APPEND mode whereby thw writer looks for the end of the file before every write? While waiting for the proper IO refactor I've been coding a csv->Inference Data converter that watches the csvs as they're created, grabs any new content then runs truncate -s 0 on the file to save on disk space. I can obviously skip the truncation part if it turns out that the csv writer here can't do O_APPEND or if there's a non-negligible implementation or performance cost, but thought I'd check.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The issue does not name a file, test, or entry point. Locate the CSV writer and first inspect how it opens and writes output files, then assess whether O_APPEND fits the current IO design and performance constraints. Done means the feasibility and expected behavior are established, with relevant coverage if a change is warranted.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
data
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.