[Bug]: buffer jumping using ordinal numbers

### Is there an existing issue for this?

- [X] I have searched the existing issues

### What happened?

I'd like to use ordinal numbers on buffer titles, but example mappings `lua require("bufferline").go_to_buffer(1, true)` in the doc seems not to work with jumping with ordinal numbers. It just does the same thing as `BufferLineGoToBuffer 1`.
For example, I opened file2 first, it gets an ordinal number 1. Then I opened file1, which gets 2 and show on the left of file2 since I configured them to be sorted by directory.
When I press leader1, it goes to file1.

### What did you expect to happen?

It should go to file2 with number 1.

### Config
```lua
require('bufferline').setup {
options = {
mode = "buffers",
numbers = function(opts)
local tmpid = opts.ordinal > 9 and 10 or opts.ordinal
local icons = {"", "", "", "", "", "", "", "", "", ""}
return icons[tmpid]
end,
-- unrelated configs omitted
sort_by = 'directory'
},
}

vim.keymap.set('n', '1', 'lua require("bufferline").go_to_buffer(1, true)', {silent = true})
vim.keymap.set('n', '2', 'lua require("bufferline").go_to_buffer(2, true)', {silent = true})
vim.keymap.set('n', '3', 'lua require("bufferline").go_to_buffer(3, true)', {silent = true})
vim.keymap.set('n', '4', 'lua require("bufferline").go_to_buffer(4, true)', {silent = true})
vim.keymap.set('n', '5', 'lua require("bufferline").go_to_buffer(5, true)', {silent = true})
vim.keymap.set('n', '6', 'lua require("bufferline").go_to_buffer(6, true)', {silent = true})
vim.keymap.set('n', '7', 'lua require("bufferline").go_to_buffer(7, true)', {silent = true})
vim.keymap.set('n', '8', 'lua require("bufferline").go_to_buffer(8, true)', {silent = true})
vim.keymap.set('n', '9', 'lua require("bufferline").go_to_buffer(9, true)', {silent = true})
```

### Additional Information

...

### commit

_No response_