AttributeError: `EasyDict` object has no attribute `moment_layers` when loading pre-trained CG-DETR checkpoints.
- Dominant language
- Python
- Stars
- 269
- Forks
- 20
- Avg merge
- 7h 3m
- Merged PRs (30d)
- 2
Description
Loading a pre-trained CG-DETR checkpoint from the [uploaded pre-trained weights](https://drive.google.com/file/d/1jxs_bvwttXTF9Lk3aKLohkqfYOonLyrO/view?usp=sharing) fails with the following error:
```
AttributeError: 'EasyDict' object has no attribute 'moment_layers'
```
**To reproduce**
Download pre-trained model weights from the [google drive URL](https://drive.google.com/file/d/1jxs_bvwttXTF9Lk3aKLohkqfYOonLyrO/view?usp=sharing) mentioned in the Readme and then load a CG-DETR checkpoint using `lighthouse.models.CGDETRPredictor`.
```python
from lighthouse.models import CGDETRPredictor
model = CGDETRPredictor('results/cg_detr/qvhighlight/clip/best.ckpt', device='cpu', feature_name='clip')
```
**Full traceback**
```
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
Cell In[19], line 1
----> 1 model = CGDETRPredictor('results/cg_detr/qvhighlight/clip/best.ckpt', device='cpu',
2 feature_name='clip')
File .../.venv/lib/python3.9/site-packages/lighthouse/models.py:363, in CGDETRPredictor.__init__(self, ckpt_path, device, feature_name, slowfast_path, pann_path)
355 def __init__(
356 self,
357 ckpt_path: str,
(...)
361 pann_path: Optional[str] = None
362 ) -> None:
--> 363 super().__init__('cg_detr', ckpt_path, device,
364 feature_name, slowfast_path, pann_path)
File .../.venv/lib/python3.9/site-packages/lighthouse/models.py:85, in BasePredictor.__init__(self, model_name, ckpt_path, device, feature_name, slowfast_path, pann_path)
81 self._audio_encoder = self._initialize_audio_encoder(feature_name, pann_path)
83 self._text_encoder: TextEncoder = self._initialize_text_encoder(feature_name)
---> 85 self._model: torch.nn.Module = self._initialize_model(args, model_name)
86 self._load_weights(ckpt['model'])
88 self._feature_name: str = feature_name
File .../.venv/lib/python3.9/site-packages/lighthouse/models.py:107, in BasePredictor._initialize_model(self, args, model_name)
96 model_builders = {
97 'moment_detr': build_model_moment_detr,
98 'qd_detr': build_model_qd_detr,
(...)
103 'taskweave': build_model_task_weave
104 }
106 if model_name in model_builders:
--> 107 model, _ = model_builders[model_name](args)
108 return model
109 else:
File .../.venv/lib/python3.9/site-packages/lighthouse/common/cg_detr.py:998, in build_model(args)
995 def build_model(args):
996 device = torch.device(args.device)
--> 998 transformer = build_transformer(args)
999 position_embedding, txt_position_embedding = build_position_encoding(args)
1001 model = CGDETR(
1002 transformer,
1003 position_embedding,
(...)
1013 args=args
1014 )
File .../.venv/lib/python3.9/site-packages/lighthouse/common/cg_detr_transformer.py:883, in build_transformer(args)
882 def build_transformer(args):
--> 883 return Transformer(
884 d_model=args.hidden_dim,
885 dropout=args.dropout,
886 nhead=args.nheads,
887 dim_feedforward=args.dim_feedforward,
888 num_encoder_layers=args.enc_layers,
889 num_decoder_layers=args.dec_layers,
890 normalize_before=False,
891 return_intermediate_dec=True,
892 activation='prelu',
893 args=args
894 )
File .../.venv/lib/python3.9/site-packages/lighthouse/common/cg_detr_transformer.py:150, in Transformer.__init__(self, d_model, nhead, num_queries, num_encoder_layers, num_decoder_layers, dim_feedforward, dropout, activation, normalize_before, return_intermediate_dec, query_dim, keep_query_pos, query_scale_type, num_patterns, modulate_t_attn, bbox_embed_diff_each_layer, args)
147 mcls_encoder_layer = TransformerEncoderLayer(d_model, nhead, dim_feedforward,
148 dropout, activation, normalize_before)
149 mcls_encoder_norm = nn.LayerNorm(d_model) if normalize_before else None
--> 150 self.mcls_encoder = TransformerEncoder(mcls_encoder_layer, args.moment_layers, mcls_encoder_norm)
152 t2v_encoder_layer = T2V_TransformerEncoderLayer(d_model, nhead, dim_feedforward,
153 dropout, activation, normalize_before, self.args.num_dummies)
154 encoder_norm = nn.LayerNorm(d_model) if normalize_before else None
AttributeError: 'EasyDict' object has no attribute 'moment_layers'
```
The checkpoint doesn't contain the `moment_layers` key although the code expects it to. Maybe the checkpoint was generated on an older version where this attribute wasn't saved?
I attempted to work around this by patching the checkpoint to include `moment_layers`:
```
if not hasattr(ckpt['opt'], 'moment_layers'):
ckpt['opt']['moment_layers'] = 1 # Default value for moment_layers
torch.save(ckpt, 'results/cg_detr/qvhighlight/clip_slowfast/best_patched.ckpt')
```
Loading the model with the patched checkpoint gave a new error: `AttributeError: 'EasyDict' object has no attribute 'num_dummies`.
It seems like these checkpoints are outdated/incompatible with the current Lighthouse version and should either be replaced by new ones or the code made compatible with these checkpoints. I might've just misunderstood the inference instructions though and I'm doing something wrong on my end, in which case sorry for the issue!
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by tracing checkpoint loading in lighthouse/models.py, then compare the model construction paths in lighthouse/common/cg_detr.py and lighthouse/common/cg_detr_transformer.py with the saved checkpoint options. Check which configuration keys are required by the current code and absent from the provided weights. Done means the README-linked CG-DETR checkpoint loads through CGDETRPredictor without missing-attribute errors.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- computer-vision, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100