openresty / openresty/lua-nginx-module

shdict "safe_set" may still remove data

Open
#1,365 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C
Stars
11.8k
Forks
2.1k
Avg merge
6h 1m
Merged PRs (30d)
6

Description

When using safe_set on a shared dict, setting a key to a large value (that won't fit in the shdict), will result in the key being removed completely from the shdict.

My assumption had been that safe_set would never override items when running out of memory, but upon closer inspection of the documentation I'm not entirely sure whether this is a bug or intended behavior (perhaps safe_set only prevents other items from being removed). If this is the intended behavior of safe_set, then maybe the documentation should be updated to be more explicit about this specific situation? I'd be happy to submit a documentation pull request if this is the case.

Here's a quick resty example (using OpenResty 1.13.6.2) that shows how setting the foo key on a shdict to something that's too large results in the removal of the original foo value (I would expect that the second safe_set call would fail, but I had expected that the original 128 byte value for foo would be retained instead of replacing it with nil):

$ resty --shdict 'testing 16k' -e '
local random = require "resty.random"

local ok, err = ngx.shared.testing:safe_set("foo", random.bytes(128))
print("1. safe_set ok: ", ok)
print("1. safe_set err: ", err)
print("1. get type: ", type(ngx.shared.testing:get("foo")))

print("") 

local ok, err = ngx.shared.testing:safe_set("foo", random.bytes(1024 * 17))
print("2. safe_set ok: ", ok)
print("2. safe_set err: ", err)
print("2. get type: ", type(ngx.shared.testing:get("foo")))'


1. safe_set ok: true
1. safe_set err: 
1. get type: string

2. safe_set ok: false
2. safe_set err: no memory
2. get type: nil

Thanks!

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the documented ngx.shared.DICT:safe_set behavior with the resty command and the oversized value shown in the issue. Compare the observed result with the safe_set documentation and determine whether retaining the existing value is intended; done means the behavior is clarified through the appropriate implementation, test, or documentation change.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, lua
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.