how to force to use a specific identity from ssh agent?

I've tried to run
`gradle release -Prelease.versionIncrementer=incrementPatch`
on a remote machine where I'm loggedin with ssh and forwarded agent

`ssh-add -l` shows my identities but when I run the mentioned command it ends with.

```
> Task :some-fancy-lib:verifyRelease
Looking for uncommitted changes..
Checking if branch is ahead of remote..
Checking for snapshot versions..

> Task :some-fancy-lib:release FAILED
Creating tag: v1.3.2
Pushing all to remote: origin
Exception occurred during push: org.eclipse.jgit.api.errors.TransportException: ssh://git@gitlab.fancycompany.de:2222/development/libs/some-fancy-lib.git: Auth fail for methods 'publickey'

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':some-fancy-lib:release'.
> org.eclipse.jgit.api.errors.TransportException: ssh://git@gitlab.fancycompany.de:2222/development/libs/some-fancy-lib.git: Auth fail for methods 'publickey'

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
> Get more help at https://help.gradle.org.

BUILD FAILED in 861ms
2 actionable tasks: 2 executed

```

copying my private key to the remote-machine and setting
```
release.customKeyFile=/home/fancyuser/.ssh/id_ed25519
release.customKeyPassword=
```
works, but I don't want to distribute my private key on each machine.

Is there something I can do to force the usage of a specific forwarded identity?