tensorflow / tensorflow/recommenders
Sample weights for both Retrieval and Ranking
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2k
- Forks
- 300
- PR merge metrics
- No merged PRs in 30d
Description
I'd like to ask about using the 'sample_weight' argument for both the Retrieval and the Ranking Tasks' call methods.
-
For both models, does it make sense to apply higher weights for samples which are more recent and smaller weights to samples that are older, individually for each user (meaning that the weights are computed individually for each user)?
-
For the Ranking model, we have an imbalanced dataset (clicks on the webage, and we control the imbalance ratio) and we already tried to use the
'sample_weight'parameter, and we were surprised by the fact that not just the loss is weighted, but also the metrics. Based on this tutorial, we supposed that the'sample_weight'parameter is used here to tackle class imbalance, similar to the'class_weight'parameter when calling the Keras model.fit() method, by weighting the loss function, but not the metrics (source code). Could you elaborate on why the metrics are weighted? According to my experience weighting the loss function should have an indirect effect on the metrics. -
Should we use
'sample_weight'-s for both the train and the validation dataset? If we don't use it for both, then there will be a huge gap due to the metrics weighting between the two resulting metrics. Otherwise the weights will be applied to the validation samples as well, which is not correct I think. -
Retrieval Task - Could you mention examples for these parameters and use-cases?
4.1batch_metrics: Optional[List[tf.keras.metrics.Metric]] = None
4.2loss_metrics: Optional[List[tf.keras.metrics.Metric]] = None -
Ranking Task - Could you mention examples for these parameters and use-cases?
5.1.prediction_metrics: Optional[List[tf.keras.metrics.Metric]] = None
5.2label_metrics: Optional[List[tf.keras.metrics.Metric]] = None
5.3.loss_metrics: Optional[List[tf.keras.metrics.Metric]] = None
I would greatly appreciate some help from the community from those who are more experienced in the above questions.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the Retrieval and Ranking API documentation and the referenced tensorflow_recommenders/tasks/ranking.py source. Trace how sample_weight and the listed metric parameters are handled, then document the requested use cases and train/validation guidance. Done means the questions have clear, supported explanations and examples in the relevant documentation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- documentation, machine-learning
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100