apache / apache/apisix

help request: Unclear if memory leak or something else in luajit process

Open
#12,928 4 comments 0 reactions 0 assignees View on GitHub
question
Dominant language
Lua
Stars
17.1k
Forks
2.9k
Avg merge
3d 18h
Merged PRs (30d)
62

Description

### Description

Running APISIX in Kubernetes, have been noticing memory going up constantly, and when it gets to 70-75%, Kubernetes kills the pods. In next image, first sawtooth at 70% is Kube killing the pods, the other ones are me restarting the pods.

Image

Tried to reproduce locally, ran the same version with same configurations in my Docker, locally, and created a (K6) script to exercise one of the routes which I think might be related to the issue. Same request (route, payload), over and over, for 10 min, simulating 20 users. Exec-ed in the docker container, ran top command, noticed a raise in memory after each run.

**First run of 10min**

Image

...
**Second run of 10min**

Image

...
**Fourth run of 10min**

Image

...
Looks like luajit went from 0.8% to 4%, which COULD explain the memory going up constantly. Digging around, found 2 links which might pertain to this issue:
1. https://github.com/apache/apisix/issues/11680 - this suggests setting a "memory limit" ("It does not appear to be a leak but rather the allocator behavior. Once a memory limit was set, this behavior was not observed again in 3.13"), any idea which memory limit for allocator it's talking about?
2. https://blog.openresty.com/en/luajit-plus/ - this one seems to be a plug for some "LuaJIT-plus", unclear if useful

Any help on dealing with this?
Thanks!

### Environment

- APISIX version (run `apisix version`): 3.13
- Operating system (run `uname -a`): Linux 0abad5fe5166 6.10.14-linuxkit #1 SMP Tue Apr 15 16:00:54 UTC 2025 aarch64 GNU/Linux
- OpenResty / Nginx version (run `openresty -V` or `nginx -V`): nginx version: openresty/1.27.1.2
- etcd version, if relevant (run `curl http://127.0.0.1:9090/v1/server_info`): N/A
- APISIX Dashboard version, if relevant: N/A
- Plugin runner version, for issues related to plugin runners: N/A
- LuaRocks version, for installation issues (run `luarocks --version`): N/A

Contributor guide

Open the contributing guide

Research direction

Start by reproducing APISIX 3.13 in the reported Docker or Kubernetes setup with the mentioned K6 route workload, then compare LuaJIT/OpenResty memory across repeated runs. Use `apisix version` and `openresty -V` as the environment entry points, with linked issue #11680 as investigation context. Done means determining whether this is a leak or allocator behavior and documenting a confirmed mitigation or next debugging step.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, kubernetes, lua
Domain
api, backend, infrastructure, performance
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.