Training SAM3 but freezing specific layers. Want to finetune on new data but keep modifiers

I’m looking to fine-tune SAM 3 on a custom dataset of industrial objects that the base model doesn't recognize well (specifically doesn't recognize the word). I have a question about how finetuning affects the model's ability to understand modifiers (adjectives) that aren't in my training data.

My training data is labeled with simple noun phrases (e.g., just forklift or pallet_jack). The labels don't describe the object's state or color.
If I fine-tune on this data, will the model lose its ability to understand adjectives like "yellow" vs "orange" because I'm never reinforcing them during training? Is there a current method to freeze certain parts to maintain SAM3's ability to use adjectives and modifiers like color, shape, size etc.

As a simplified example, if we were using the forklift example, my dataset would have both of these objects labeled as just forklift

Start with the repository's fine-tuning code and example notebooks to determine whether freezing specific layers is supported. Compare the available training controls with the requested preservation of modifier understanding; done would require a clear maintainer answer or an explicitly scoped change.

Written by the indexing model from the issue text.