Incoporate Time-Varying Data in "CoxCCDataset" function

Hi Havakv, great to have a meeting with you yesterday, you showed us the right direction to incorporate time-varying survival data.
I am trying to incorporate time-varying survival data (or counting process data) inside your pycox package using your built-in sample data "metabric"

1, I created two new variables representing the start of follow-up and end of follow-up for time-varying data, I used your sample data metabric and extended it to time-varying survival data:
df_train = metabric.read_df()
df_train["START"] = 0
df_train["STOP"] = df_train["duration"]

2, follow your suggestion, I am trying to modify the case-control selection part to select control based on start and stop time rather than the duration column:

class CoxCCDataset_V2(torch.utils.data.Dataset):
def __init__(self, input, durations, events, n_control=1):

df_train_target = pd.DataFrame(dict(duration=durations, event=events, START=START, STOP=STOP))
new_df = pd.DataFrame() # dataframe for matched controls
case_df = df_train_target.loc[lambda x: x['event'] == 1]['duration'] # dataframe for cases
for i in range(0, len(df_train_target)):
if (df_train_target.iloc[i]['event'] == 1):
control_df = pd.DataFrame() # dataframe for controls
control_df = df_train_target[df_train_target['event'] == 0]
control_df = control_df[control_df['START'] <= df_train_target.iloc[i]['STOP']]
control_df = control_df[control_df['STOP'] >= df_train_target.iloc[i]['STOP']] # interval filter
control = control_df.sample()
new_df = new_df.append(control)
return new_df

self.durations = df_train_target.loc[lambda x: x['event'] == 1]['duration']
self.input = tt.tuplefy(input)
assert type(self.durations) is pd.Series
self.n_control = n_control

def __getitem__(self, index):
if (not hasattr(index, '__iter__')) and (type(index) is not slice):
index = [index]
fails = self.case_df.loc[lambda x: x['event'] == 1]['duration'].iloc[index]
x_case = self.input.iloc[fails.index]
non_fails = self.new_df.loc[lambda x: x['event'] == 0]['duration'].iloc[index]
x_control = self.input.iloc[non_fails.index]
return tt.tuplefy(x_case, x_control).to_tensor()

def __len__(self):
return len(self.durations)

I am not familiar PyTorch or Torch, can't debug which part that I wrote is wrong. If you have time, can you check the function above? Thanks!

Start at the existing CoxCCDataset entry point and the metabric sample-data path, then compare their current case-control selection with the proposed START/STOP filtering. Done means the package has a defined, working approach for time-varying survival data and the supplied example no longer requires unresolved PyTorch debugging.

Written by the indexing model from the issue text.