thuml / thuml/Brep2Shape

Finetuning loader requires pretraining reconstruction targets (uv_face_points/uv_edge_points) that the CE loss never uses

Open Beginner friendly
#7 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
11
Forks
3
PR merge metrics
No merged PRs in 30d

Description

Summary

The finetuning data loader hard-requires the pretraining reconstruction targets
(uv_face_points and uv_edge_points) for every sample, including classification
and segmentation runs, even though the downstream cross-entropy loss never reads
them. This makes preparing a new labeled downstream dataset harder than it needs to
be: you must run the pretraining target-sampling step to attach reconstruction
targets to data that will only ever be used for a class/segmentation head.

Where

datasets/finetuning_dataset.py:

  • load_face lists uv_face_points in required_keys and raises KeyError if it
    is absent.
  • load_one_sample additionally asserts the shapes:
    • face["uv_face_points"].shape == (num_faces, 3, 3, 3)
    • topo["uv_edge_points"].shape == (num_edges, 3, 3)
      and raises ValueError otherwise.

The finetune objectives in classification.py and segmentation.py compute
F.cross_entropy(logits, labels) and never consume uv_face_points or
uv_edge_points.

Why it matters

When bringing a new labeled dataset to the finetune loader (per-face labels for
segmentation, or a class label per solid), the face and topo files produced by the
tokenizer / topology step do not carry uv_face_points / uv_edge_points; those
are attached by the pretraining target-sampling stage. As a result the loader
rejects otherwise-complete finetune records with a KeyError/ValueError for a
target the task does not use. In practice this forces users to graft or recompute
reconstruction targets purely to satisfy a validation check.

Suggestion

Gate the uv_face_points / uv_edge_points requirement on the task. For
classification and segmentation, treat them as optional (skip the key check and the
shape assertions when absent), and require them only for objectives that actually
use them. A minimal version is to move those two keys out of the unconditional
required_keys set and behind an if "uv_face_points" in labels: shape check.

Happy to send a small PR if that direction is agreeable.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in datasets/finetuning_dataset.py by tracing load_face and load_one_sample, then compare their checks with the objectives in classification.py and segmentation.py. Confirm that classification and segmentation records without reconstruction targets load successfully, while targets are still checked when an objective uses them; exercise both missing-key and invalid-shape cases to verify the behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
76/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.