Trying to change C-k keybinding on the minibuffer doesn't work

### What did you expect to happen?

I need to change my movement keys since I do not use a standard keyboard layout, I've successfully done this in normal and visual mode, however trying it for the minibuffer the bindings seem to be ignored, for example, after pressing `M-x` the selection list pops up in the minibuffer (I think that's ivy) then pressing `C-k` should move the selection to the *next* item in the list, but actually keeps moving to the *previous* element (the default behavior).

### What actually happened?

It ignores the bindings as if they were not defined at all, `C-k` keeps moving to the previous item instead of the next one. It only happens with `C-k` though, `C-h` binding is picked up fine.

### Describe your attempts to resolve the issue

This is what I'm trying:

```elisp
(map! :map vertico-map
:g "C-k" 'next-line
:g "C-h" 'previous-line
:g "C-S-k" 'scroll-down-command
:g "C-S-h" 'scroll-up-command)

(map! :map minibuffer-inactive-mode-map
:g "C-k" 'next-line
:g "C-h" 'previous-line
:g "C-S-k" 'scroll-down-command
:g "C-S-h" 'scroll-up-command)

(define-key! :heymaps +default-minibuffer-maps "C-k" 'next-history-element)
(define-key! :heymaps +default-minibuffer-maps "C-h" 'previous-history-element)

(map! :map read-expression-map
:g "C-k" 'next-line-or-history-element
:g "C-h" 'previous-line-or-history-element)
```

### Steps to reproduce

I have a clean installation of doom with only this in the `~/.doom.d/config.el` file (comments removed):

```
;;; $DOOMDIR/config.el -*- lexical-binding: t; -*-

(setq user-full-name "John Doe"
user-mail-address "john@doe.com")

(setq doom-theme 'doom-one)

(setq display-line-numbers-type t)

(setq org-directory "~/org/")

(map! :g "C-s" 'save-buffer)

(map! :nv "k" 'evil-next-line
:nv "h" 'evil-previous-line
:nv "j" 'evil-backward-char)

(map! :map vertico-map
:g "C-k" 'next-line
:g "C-h" 'previous-line
:g "C-S-k" 'scroll-down-command
:g "C-S-h" 'scroll-up-command)

(map! :map minibuffer-inactive-mode-map
:g "C-k" 'next-line
:g "C-h" 'previous-line
:g "C-S-k" 'scroll-down-command
:g "C-S-h" 'scroll-up-command)

(define-key! :heymaps +default-minibuffer-maps "C-k" 'next-history-element)
(define-key! :heymaps +default-minibuffer-maps "C-h" 'previous-history-element)

(map! :map read-expression-map
:g "C-k" 'next-line-or-history-element
:g "C-h" 'previous-line-or-history-element)
```

### System Information

https://pastebin.com/u2uCizBq