Autoconfiguration does not detect vdirsyncer

I installed vdirsyncer in ~/.local/programs/vdirsyncer, its config-file is at ~/.config/vdirsyncer/config (with environment-variable set) and status_path = "~/.local/share/vdirsyncer/" set in the config-file (so the homedirectory doesn't get so cluttered)
I followed the clean-hard-way installation-method https://vdirsyncer.pimutils.org/en/stable/installation.html#the-clean-hard-way.

$ echo $VDIRSYNCER_CONFIG
/home/harald/.config/vdirsyncer/config
$ ls -al /home/harald/.config/vdirsyncer/config
-rw------- 1 harald harald 2322 Sep  3 14:40 /home/harald/.config/vdirsyncer/config

khal is installed at ~/.local/programs/khal using

$ virtualenv -p python3.7 ~/.local/programs/khal
$ ~/.local/programs/khal/bin/pip3.7 install khal

It's config is at ~/.config/khal

When running khal configure vdirsyncer cannot be found:

$ khal configure
...
If you use vdirsyncer to sync with CalDAV servers, we can try to load its config file and add your calendars to khal's config.
Should we try to load vdirsyncer's config? [Y/n]: y
Sorry, cannot import vdirsyncer. Please make sure you have it installed.
...

$ echo $VDIRSYNCER_CONFIG
/home/harald/.config/vdirsyncer/config

Start by reproducing khal configure with vdirsyncer installed under ~/.local/programs/vdirsyncer and VDIRSYNCER_CONFIG pointing to ~/.config/vdirsyncer/config. Trace the vdirsyncer import and configuration discovery used by the command; done means khal detects the installation and offers calendars from that configuration.