face recognition error

I created an actor database with hundreds of people and used a TV drama to extract faces for recognition. When I did face recognition on continuous frames of the same person, I found that the first few frames were correctly recognized, but then a mouth movement was recognized as another character, and the similarity was above 0.99. I set the parameter det_prob_threshold to 0.8 when creating the template and performing face recognition. I also tried adjusting it from 0.5 to 0.95, but the problem still persisted. Are there any other parameters that can be adjusted?