xmaint_allowed_ips_list redis WRONGTYPE error

Chef Server Version

15.10.66

Platform Details

Rocky 9, AWS EC2, t3a.large

Configuration

Standalone, recently upgraded to 15.10.66.

Scenario

chef-serverctl reconfigure

Chef server normal operation breaks because the redis key xmaint_allowed_ips_list is the wrong type. Error messages can be found in nginx error.log:

2025/10/24 18:40:07 [error] 622419#0: *18 [lua] config.lua:101: redis_fetch_set(): Redis read error retrieving xmaint_allowed_ips_list: WRONGTYPE Operation against a key holding the wrong kind of value, client: 172.20.22.146, server: chef-007f37.mycorp.corp, request: "GET /organizations/mycorp/environments/_default HTTP/1.1", host: "chef.mycorp.com"

It looks like this is happening because chef-serverctl reconfigure is writing a string type value to the xmaint_allowed_ips_list redis key in server-ctl-cookbooks/infra-server/recipes/redis_lb.rb#L124-L125:

redis.set('xdl_defaults', xdl.to_json)
redis.set('xmaint_allowed_ips_list', xmaint_allowed_ips_list.to_json)

.to_json here is turning the ruby objects into a json strings, and writing that string value to redis.

However, in chef-server-ctl/plugins/maintenance.rb, this is being treated as a set, not a string:

redis.sadd("xmaint_allowed_ips_list", ip)

The same goes for nginx/scripts/config.lua.erb:

ok, allowed_ips_list = redis_fetch_set("xmaint_allowed_ips_list")

It's this line I think that's throwing the error. This line is correct. The data that's getting written to redis above is what's wrong.

You can confirm that this is a string type and that the error message is correct:

$ keydb-cli -p 16379
127.0.0.1:16379> TYPE xmaint_allowed_ips_list
string

The (very) temporary fix (ie proof of the issue/solution) is to re-write the key as the correct type.

127.0.0.1:16379> del xmaint_allowed_ips_list
(integer) 1
127.0.0.1:16379> sadd xmaint_allowed_ips_list '127.0.0.1'
(integer) 1
127.0.0.1:16379> TYPE xmaint_allowed_ips_list
set

Now, chef server runs and is happy.

This problem likely also affects the redis key xdl_defaults, which is supposed to be a hash (Mash in chef terms, not sure of the redis-specific term) but is also being written as a string because of the to_json. However, so far it doesn't seem to matter and that chef does not appear to be reading this value.

I believe the more permanent fix is to remove the to_json call and instead iterate over the values in the xmaint_allowed_ips_list array, calling sadd for each array value. There may be a more direct way to write an array as a type set to redis that I'm not aware of.

Steps to Reproduce

• chef-serverctl reconfigure
• keydb-cli -p 16379
  • TYPE xmaint_allowed_ips_list will return string

Expected Result

The type for xmaint_allowed_ips_list should be set

Actual Result

The chef server will not correctly respond to requests from chef clients, but will throw an error in nginx error.log:

2025/10/24 18:40:07 [error] 622419#0: *18 [lua] config.lua:101: redis_fetch_set(): Redis read error retrieving xmaint_allowed_ips_list: WRONGTYPE Operation against a key holding the wrong kind of value, client: 172.20.22.146, server: chef-007f37.mycorp.corp, request: "GET /organizations/mycorp/environments/_default HTTP/1.1", host: "chef.mycorp.com"

Start with omnibus/files/server-ctl-cookbooks/infra-server/recipes/redis_lb.rb around lines 124-125, then compare its Redis writes with src/chef-server-ctl/plugins/maintenance.rb and nginx/scripts/config.lua.erb. Run chef-serverctl reconfigure and inspect xmaint_allowed_ips_list with keydb-cli TYPE. Done means the key is stored as a set and Chef Server requests no longer produce the Redis WRONGTYPE error.