bytedance / bytedance/LPCD

LinkBuilder ignores non-temporal graph edge scales

Open
#1 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1
Forks
1
PR merge metrics
No merged PRs in 30d

Description

## Summary

`LinkBuilder.forward` only assigns the temporal-edge term to `adj_filtered`. The user, anchor-user, and fallback/malicious terms are written as separate expression statements, so their learnable scale parameters do not affect the returned adjacency.

This occurs in both `model/ours/ACMIL_v2.py` and `model/ours/patchMIL_v20.py` at commit `a0b47382f08b108a17a39b343c4e58dbf2852db3`.

## Minimal reproduction

```python
import torch
from model.ours.ACMIL_v2 import LinkBuilder

layer = LinkBuilder(hidden_dim=2, adj_scale_init=0.5)
patches = torch.tensor([[[1., 0.], [2., 0.], [3., 0.], [4., 0.]]])
valid = torch.ones(1, 4, dtype=torch.bool)
graph = torch.zeros(1, 4, 4, 3, dtype=torch.bool)
graph[0, 0, 1, 0] = True # temporal
graph[0, 0, 2, 1] = True # same user
graph[0, 0, 3, 2] = True # anchor-user

layer(patches, valid, graph).square().sum().backward()
for name, parameter in layer.named_parameters():
print(name, parameter.grad)
```

Actual result: `scale_factor_time_adj` has a gradient, while `scale_factor_user_adj`, `scale_factor_anchor_user_adj`, and `scale_factor_malicious_adj` all have `None` gradients. The same result occurs with `model.ours.patchMIL_v20.LinkBuilder`.

## Expected behavior

All four relation-specific terms should participate in the adjacency calculation, so each scale can influence the result and receive gradients when its edge category is present.

## Impact

Graph-aware attention currently ignores same-user, anchor-user, and fallback relationships despite defining trainable scales for them. Training can therefore only learn the temporal-edge scale.

## Suggested fix

Wrap the four weighted terms in a single parenthesized expression in both implementations and add a regression test that exercises every edge category.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with LinkBuilder.forward in model/ours/ACMIL_v2.py and model/ours/patchMIL_v20.py at commit a0b47382f08b108a17a39b343c4e58dbf2852db3. Run the minimal PyTorch reproduction, then add a regression test covering temporal, same-user, anchor-user, and fallback/malicious edges. Done means all four relation-specific scale parameters affect the adjacency and receive gradients when their edge category is present.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning, testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.