tidymodels / tidymodels/workflows

plumbing post-processing for survival analysis

Open
#229 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

feature
Dominant language
R
Stars
211
Forks
26
Avg merge
1h 58m
Merged PRs (30d)
1

Description

From Hannah in the linked discussion:

Okay so here are my thoughts on what plumbing post-processing for survival analysis would need.

Basic assumptions

  • We might tailor both predictions of survival time and predictions of survival probability.
  • If we tailor survival probability, we will do so at a specific single time point (hello eval_time our old friend).

I have not yet validated either assumption. Max, do you have a sense of whether they are valid?

For the implementation, this implies

  • How/where do we specify the predictions?
  • How/where do we specify eval_time?

Specifying the predictions

  • tailor() could take survival time .pred_time via the estimate argument -- the documentation already suggest that.
  • tailor() could take survival probability in the tibble form of .pred (containing .eval_time and .pred_survival) via the probabilities argument.

Specifying eval_time

  • [tailor] tailor() would drop time (the documentation for that as "the predicted event time" contradicts the documentation for estimate listing .pred_time) and instead take eval_time. The alternative would be to derive it from .pred within a given postprocessing operation and default to the first value if we need a single one. Either way, time would disappear.
  • [workflows] We currently can pass eval_time to predict() and augment() methods for workflows but since we currently don't need it earlier, there are not arguments for eval time to the fit() method for workflows or the specification via workflow().
    • If the post-processing operation doesn't need estimation, only predict.workflow() having an eval_time argument should be fine.
    • If it does need estimation, it could come from the tailor() specification.
  • [tune] The tuning functions have an eval_time argument which is required for dynamic and integrated survival metrics. If we need a single eval time to optimize for, we use the first one.
    • If a user includes a tailor in the workflow to be tuned, it would be nice to not make them specify eval_time twice: when making the tailor() and when calling the tuning function.
    • We could pass it through workflows (and make it gain an argument) or make a function to update the tailor. I lean towards updating the tailor.

Originally posted by @hfrick in https://github.com/tidymodels/workflows/pull/225#pullrequestreview-2035569467

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the linked discussion and trace the APIs named in the issue: tailor(), predict.workflow(), augment(), fit(), workflow(), and the tuning functions. Map how survival-time and survival-probability predictions and eval_time currently flow through these entry points. Done means the unresolved assumptions and ownership of eval_time are settled and the required cross-package behavior is implemented consistently.

Written by the indexing model from the issue text.

Assessment

Tech stack
r
Domain
backend-api-design
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.