neurostuff / neurostuff/autonima
Unresolvable table_id aborts the whole run instead of degrading one analysis's metadata
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4
- Forks
- 1
- Avg merge
- 12h 56m
- Merged PRs (30d)
- 2
Description
A run aborts outright when an analysis carries a table_id that cannot be resolved against its study:
autonima.pipeline - ERROR - Pipeline failed: Analysis 19826471_analysis_0 has table_id
'pone-0007402-t001' but study 19826471 has no activation_tables
annotation/processor.py:636-651 raises ValueError in two adjacent cases — the study has no activation_tables at all, and the study has tables but none matching the id.
Why this is disproportionate
Both raises sit inside _extract_analysis_metadata, whose job is to attach table_id, table_caption and table_footer as optional context for the annotation prompt. Losing a caption should degrade one analysis's metadata. Instead it terminates a pipeline that may have hours of screening and parsing behind it, and the failure arrives at the annotation stage — long after the work that would have to be redone.
The two cases are also not equally alarming and probably deserve different handling. "Study has no tables at all" usually means the study's retrieval or parsing record has moved on; "study has tables but not this one" is closer to a genuine referential inconsistency worth surfacing loudly.
How it comes up
Hit while re-running a project end-to-end against a partially rebuilt corpus (--clear-cache abstract fulltext parsing annotation, search and retrieval reused). Re-running abstract screening changed the included set, which forced retrieval to re-run, which left some studies without the activation_tables their analyses still referenced. The inconsistency is real; aborting on it is the problem.
This is reachable by any partial re-run where retrieval and parsing end up out of step, which is exactly the situation the incremental cache is designed to support.
Suggested fix
Log a warning, leave table_id/table_caption/table_footer unset for that analysis, and continue. Optionally count these at the end of the stage so they are visible rather than buried. If a hard failure is wanted for the mismatched-id case specifically, make it a config option rather than the default, and raise it early — a consistency check after parsing would surface the problem before the expensive stage rather than during it.
Related
Adjacent to #61 (sparse analysis names), which is the same shape: annotation metadata being absent should weaken a decision, not stop the run.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in annotation/processor.py:636-651 and trace _extract_analysis_metadata, then reproduce the partial rerun with --clear-cache abstract fulltext parsing annotation. Done means an unresolved table_id no longer aborts the run, missing metadata remains unset for that analysis, and the inconsistency is surfaced through a warning or stage summary.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 75/100