`git checkout` to different branch that have encrypted file failed

```
> git checkout feature/xxx
git-crypt: error: encrypted file has been tampered with!
error: external filter "git-crypt" smudge failed 1
error: external filter "git-crypt" smudge failed
fatal: /path/to/file/cacert.pem: smudge filter git-crypt failed
```

I have some encrypted file in this `feature/xxx` branch that doesn't exists in my current branch, when i try to checkout to it, it seems there is a failure on filter process of git-crypt. Has anyone encountered this before?