[Bug]: BufferLineGoToBuffer <number> doesn't go to it's own numbering system. It uses only top visible buffers

### Is there an existing issue for this?

- [x] I have searched the existing issues

### What happened?


Here as you see ther is multiple buffers. Some ar on the <- and some are on the ->
But i'f i'm on the 30th and i go to 2, it will got o the second that is visible on screen.
For me, only 5 buffers can be visible at once, so buffer goto can never move by more then 5.

### What did you expect to happen?

:BufferLineGoTo 3
when you are on the 30th buffer should properly goto it, the buffer with
3.

### Config

local function goto_buffer(_) end
goto_buffer = function(buf_num) vim.cmd("BufferLineGoToBuffer " .. buf_num) end

-- # leader# leadern n (search strings)
for i = 1, 9 do
keymap.set("n", "" .. i, function() goto_buffer(i) end, opts("Go to buffer " .. i))
end
-- Bind 0 to goto 10for buffer switching
keymap.set("n", "0", function() goto_buffer(10) end, opts("Go to buffer 10"))

----
Bufferline config:

```lua
vim.opt.termguicolors = true
local bufremove = require("mini.bufremove") -- Load once

local bufferline = require("bufferline")

bufferline.setup({
options = {
mode = "buffers", -- "tabs" to show only tabs
-- hover = {
-- enabled = true,
-- delay = 200,
-- reveal = { "close" },
-- },
show_buffer_close_icons = true, -- Hide close icons by defaul

numbers = "ordinal", -- Show buffer numbers
close_command = function(bufnum) bufremove.delete(bufnum, false) end,
right_mouse_command = function(bufnum) bufremove.delete(bufnum, false) end,
left_mouse_command = "buffer %d", -- Switch buffer on click
middle_mouse_command = nil, -- No middle-click behavior
indicator = { style = "underline" }, -- Style for active buffer indicator
buffer_close_icon = "", -- Icon for closing buffer
modified_icon = "●", -- Icon for unsaved changes
close_icon = "", -- Icon for closing all buffers
left_trunc_marker = "", -- Indicator for left truncation
right_trunc_marker = "", -- Indicator for right truncation
separator_style = "thin", -- Options: "slant", "thick", "thin"
diagnostics = "nvim_lsp", -- Show LSP errors/warnings in buffers
diagnostics_indicator = function(count, level, diagnostics_dict, context)
local s = " "
for e, n in pairs(diagnostics_dict) do
local sym = e == "error" and " " or (e == "warning" and " " or " ")
s = s .. n .. sym
end
return s
end,
custom_filter = function(buf_number, _)
-- Hide terminal buffers from appearing in bufferline
if vim.bo[buf_number].buftype ~= "terminal" then
return true
end
end,
always_show_bufferline = true, -- Always show even with 1 buffer
sort_by = "insert_at_end", -- Buffer sorting method
offsets = {
{
filetype = "NvimTree",
text = "File Explorer",
highlight = "Directory",
text_align = "center",
separator = true, -- Show a separator between nvim-tree and buffers
padding = 1, -- Space between the file tree and bufferline
},
},
},
})

```

Note that the keybinds aren't important, because i manually tested with :Goto

### Additional Information

...
Maybe keep a list of buffers and use that for goto, or hacky way: Just loop goto until its actually there,

● bufferline.nvim 0.31ms  start
dir /home/francois/.local/share/nvim/lazy/bufferline.nvim
url https://github.com/akinsho/bufferline.nvim
version 4.9.1
tag v4.9.1
branch main
commit 655133c
readme README.md
help |bufferline.nvim|

### commit

655133c