tidymodels / tidymodels/workflows
plumbing post-processing for survival analysis
Nobody has claimed this yet.
- Dominant language
- R
- Stars
- 211
- Forks
- 26
- Avg merge
- 1h 58m
- Merged PRs (30d)
- 1
Description
From Hannah in the linked discussion:
Okay so here are my thoughts on what plumbing post-processing for survival analysis would need.
Basic assumptions
- We might tailor both predictions of survival time and predictions of survival probability.
- If we tailor survival probability, we will do so at a specific single time point (hello
eval_timeour old friend).
I have not yet validated either assumption. Max, do you have a sense of whether they are valid?
For the implementation, this implies
- How/where do we specify the predictions?
- How/where do we specify
eval_time?
Specifying the predictions
tailor()could take survival time.pred_timevia theestimateargument -- the documentation already suggest that.tailor()could take survival probability in the tibble form of.pred(containing.eval_timeand.pred_survival) via theprobabilitiesargument.
Specifying eval_time
- [tailor]
tailor()would droptime(the documentation for that as "the predicted event time" contradicts the documentation forestimatelisting.pred_time) and instead takeeval_time. The alternative would be to derive it from.predwithin a given postprocessingoperationand default to the first value if we need a single one. Either way,timewould disappear. - [workflows] We currently can pass
eval_timetopredict()andaugment()methods for workflows but since we currently don't need it earlier, there are not arguments for eval time to thefit()method for workflows or the specification viaworkflow().- If the post-processing operation doesn't need estimation, only
predict.workflow()having aneval_timeargument should be fine. - If it does need estimation, it could come from the
tailor()specification.
- If the post-processing operation doesn't need estimation, only
- [tune] The tuning functions have an
eval_timeargument which is required for dynamic and integrated survival metrics. If we need a single eval time to optimize for, we use the first one.- If a user includes a tailor in the workflow to be tuned, it would be nice to not make them specify
eval_timetwice: when making thetailor()and when calling the tuning function. - We could pass it through workflows (and make it gain an argument) or make a function to update the tailor. I lean towards updating the tailor.
- If a user includes a tailor in the workflow to be tuned, it would be nice to not make them specify
Originally posted by @hfrick in https://github.com/tidymodels/workflows/pull/225#pullrequestreview-2035569467
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the linked discussion and trace the APIs named in the issue: tailor(), predict.workflow(), augment(), fit(), workflow(), and the tuning functions. Map how survival-time and survival-probability predictions and eval_time currently flow through these entry points. Done means the unresolved assumptions and ownership of eval_time are settled and the required cross-package behavior is implemented consistently.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- r
- Domain
- backend-api-design
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100