__pairs and __ipairs metamethods have incorrect annotations

How are you using the lua-language-server?

Visual Studio Code Extension (sumneko.lua)

Which OS are you using?

Windows

What is the issue affecting?

Annotations

Expected Behaviour

lua-language-server should specify correct function signatures for __pairs and __ipairs.

lua-language-server should in particular say that the __ipairs iterator generator function has 3 return values.

Following the annotation hints for __ipairs should not result in runtime errors.

Given that generic for k, v in pairs(t) --or ipairs(t) for pairs and ipairs are equivilant to:

do
  local f, s, var = pairs(t) --or ipairs(t)
  while true do
    local k, v = f(s, var)
    if k == nil then break end
    var = k
    <block>
  end
end

I believe these are the required function signatures:

__pairs = 
GEN_FUN(ITERABLE)
	RETURNS
		ITER_FUN(ITERABLE,KEY)
			RETURNS 
				KEY
				VALUE
		ITERABLE
		KEY_BEFORE_FIRST
OR nil

__ipairs = 
GEN_FUN(ITERABLE_ARRAY)
	RETURNS 
		ITER_FUN(ITERABLE_ARRAY,INDEX)
			RETURNS 
				INDEX OR nil
				VALUE
		ITERABLE_ARRAY
		INDEX_BEFORE_FIRST
OR nil

which I believe should be:

---@field __pairs (fun(t):((fun(t,k):(any,any)),any,any))|nil
---@field __ipairs (fun(t):((fun(t,k):((integer|nil),any)),any,integer))|nil

Actual Behaviour

__pairs annotation specifies the iterator function as taking a third argument, for the value. Lua versions 5.2 and 5.4 do not pass a third argument to the iterator function.

---@field __pairs (fun(t):((fun(t,k,v):any,any),any,any))|nil

__ipairs annotation specifies the iterator function as taking a third argument, for the value. Lua version 5.2 does not pass a third argument to the iterator function.

__ipairs annotation specifies the generator function as returning only 2 results. Following the annotation hinting results in nil being implicitly passed as the third argument, instead of the correct index.

---@field __ipairs (fun(t):(fun(t,k,v):(integer|nil),any))|nil

Reproduction steps

t = setmetatable({1,2,3}, {
  __ipairs = function(t) local function iter(t, k) k = k + 1 local v = t[k]; if v then return k, v end; end; return iter, t, 0 end,
})

for k, v in ipairs(t) do
  print(v)
end

(field) __ipairs: function|nil
function (t: any)
  -> fun(t: any, k: any, v: any):integer|nil
  2. any
function __ipairs(t: any)
  -> function
  2. unknown
  3. integer

Annotations specify that at most 2 return value(s) are required, found 3 returned here instead.Lua Diagnostics.(redundant-return-value)

Removing the extra return value as the annotation suggests results in :

.\sample.lua:2: attempt to perform arithmetic on local 'k' (a nil value)
stack traceback:
        .\source.txt:2: in function 'for iterator'
        .\source.txt:5: in main chunk
        [C]: in ?

Additional Notes

The example __ipairs sample is essentially what AI generated sample code looks like, so it is likely how someone would use __ipairs.

Lua 5.2 allows iteration with __ipairs to start at any point; this is useful if __ipairs is used to support non-standard ipairs iteration, such as starting from a negative index or starting at some arbitrary positive index, controlled by altering the third return value from the generator function to be a non-zero value.

Some of the __ipairs choices in the suggested annotations I made aren't technically explicitly mandated, but I feel they are in the spirit of the intended functionality; nothing stops anyone violating the implicit assumption that the keys are sequential, have numeric indexes, or they will be traversed in a given order:

t = setmetatable({a=1,b=2,c=3}, {
  __ipairs = function(t) local function iter(t, k) return next(t,k) end; return iter, t, nil end,
})

for k, v in ipairs(t) do
  print(k, v)
end

I kept my suggested annotation for __ipairs in line with the normal behaviors of ipairs when there is no __ipairs metamethod.

Log File

No response

Start with the __pairs and __ipairs annotation definitions and run the supplied Lua reproduction in the lua-language-server environment. Compare the inferred signatures with Lua 5.2 and 5.4 iteration behavior. Done means the annotations accept the three generator returns, model the iterator arguments and returns correctly, and no longer produce the shown diagnostics or runtime error.