[QST]Transformers4Rec/examples/tutorial/03-Session-based-recsys.ipynb.

❓ Questions & Help

Details

I have executed 01 and 02 successfully and in 03 as well everything before this block under
"3.2.4 Train XLNET with Side Information for Next Item Prediction" had no issues.

However this block is giving some issue :

%%time
start_time_window_index = 1
final_time_window_index = 4
for time_index in range(start_time_window_index, final_time_window_index):
    # Set data 
    time_index_train = time_index
    time_index_eval = time_index + 1
    train_paths = glob.glob(os.path.join(OUTPUT_DIR, f"{time_index_train}/train.parquet"))
    eval_paths = glob.glob(os.path.join(OUTPUT_DIR, f"{time_index_eval}/valid.parquet"))
    # Train on day related to time_index 
    print('*'*20)
    print("Launch training for day %s are:" %time_index)
    print('*'*20 + '\n')
    trainer.train_dataset_or_path = train_paths
    trainer.reset_lr_scheduler()
    trainer.train()
    trainer.state.global_step +=1
    # Evaluate on the following day
    trainer.eval_dataset_or_path = eval_paths
    train_metrics = trainer.evaluate(metric_key_prefix='eval')
    print('*'*20)
    print("Eval results for day %s are:\t" %time_index_eval)
    print('\n' + '*'*20 + '\n')
    for key in sorted(train_metrics.keys()):
        print(" %s = %s" % (key, str(train_metrics[key]))) 
    wipe_memory()

Error is :

***** Running training *****
  Num examples = 22784
  Num Epochs = 3
  Instantaneous batch size per device = 256
  Total train batch size (w. parallel, distributed & accumulation) = 256
  Gradient Accumulation steps = 1
  Total optimization steps = 267
********************
Launch training for day 1 are:
********************

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<timed exec> in <module>

[/usr/local/lib/python3.9/dist-packages/transformers/trainer.py](https://localhost:8080/#) in train(self, resume_from_checkpoint, trial, ignore_keys_for_eval, **kwargs)
   1394 
   1395             step = -1
-> 1396             for step, inputs in enumerate(epoch_iterator):
   1397 
   1398                 # Skip past any already trained steps if resuming training

17 frames
[/usr/local/lib/python3.9/dist-packages/cudf/io/dlpack.py](https://localhost:8080/#) in to_dlpack(cudf_obj)
     90     gdf = gdf.astype(dtype)
     91     arr_cupy = cp.array(df.fillna(-1).to_gpu_matrix())
---> 92 
     93 
     94     return libdlpack.to_dlpack([*gdf._columns])

interop.pyx in cudf._lib.interop.to_dlpack()

ValueError: Cannot create a DLPack tensor with null values.                 Input is required to have null count as zero.

Pls help me if i am missing anything to cross check, i am unable to figure out here

Start with Transformers4Rec/examples/tutorial/03-Session-based-recsys.ipynb and run the cells before and within “3.2.4 Train XLNET with Side Information for Next Item Prediction.” Reproduce the training loop and trace the reported cuDF DLPack null-value error back to the training or evaluation data. Done means identifying the cause and documenting or correcting a reproducible path through this block.