mechababs aggregate
- Dominant language
- Python
- Stars
- 1
- Forks
- 4
- Avg merge
- 15h 39m
- Merged PRs (30d)
- 24
Description
Produce a strawman for yarik to review:
- what data gets collected
- some cases, stuff thats not committed, (babs jobs_status.csv things)
- slurm jobs
- resource usage of slurm jobs (must be collected before slurm drops them from history)
- how that data gets rendered
Concepts to consider:
- cheap render-only mode ie from cache
- query/collect mode ie needs to discover the reality first
- look into git statusd
- where cache lives git-annex uses a special branch to record the "state of the world"
- fscacher might be helpful
Goal: Dashboard "What are we doing, and how done is it"
- list all mechababs campagins
- for a campaign, what studies are involved
- for each study
- what derivatives are complete
- do we need to more info if partially complete?
- record babs specific info, like "has these results branches"?
- these jobs are running/done/pending?
- full list of jobs with resource usage?
- interaction with "ephemeral studies", ie what if the user removes the study from the cluster?
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reviewing the existing mechababs data sources, including babs jobs_status.csv and Slurm job history, then investigate git statusd, git-annex's special branch, and fscacher. Produce a strawman for yarik covering collection, caching, rendering, campaign/study/job views, resource usage, and ephemeral studies; completion means the proposed design is ready for review.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- git, python
- Domain
- data, devtools
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 28/100