nvim-tree / nvim-tree/nvim-tree.lua

Memory leak when file watcher enabled on Windows

Open
#3,292 5 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

bug OS Windows OS Windows PowerShell performance PR please
Dominant language
Lua
Stars
8.6k
Forks
639
Avg merge
1d 14h
Merged PRs (30d)
2

Description

Description

I'm observing enourmous memory consumption by neovim caused by nvim-tree plugin when using with large directory hiearchies like C: drive as root. My findings show that the cause of that is file system watcher. I know that the plugin is documented (aka by-design) to perform poorly in such scenarios, but I have some more thoughts on this.

  1. I tried to do memory profiling of neovim in VS, and found out that most allocations occur in calls to luv_fs_event_cb which apparently puts lua callback request in a queue, and the queue is growing continuously.
  2. Browsing nvim-tree source code and monitoring logs, I found out that nvim-tree registers FS watcher for every child directory and file separately when a directory is expanded. I saw the comment that recursive events monitoring is not working for some reason (though AFAIK on windows FS watcher API works ok with hierarchies). What I think this leads to is that with huge amount of callback regisrations (I counted almost 40 000 (!) in the log for just expanding c:\Users\MyUserName) neovim api has to poll for events (via ReadDirectoryChangesW system call) for each file system item separately. Results return happens asynchronously, so callback requests are queued in parallel from multiple threads, which increases chances of queue growing faster than it can be processed.
  3. As mentioned in 2, the amount of watcher registrations does not correlate with amount of expanded direcories and visible children. In the log I see many records like "Watcher:create 'C:\Users\MyUserName\AppData\Local\Microsoft\VSApplicationInsights\vstelf'", while I just expanded "c:\Users\MyUserName". In nvim-tree code I see that children exploration goes just one level, but in some cases it goes deeper recursively.
    I don't quite understand why, by I believe this is the reason the number of watched items is so huge.
    Another clue I realized from the logs: watched directories are refreshed on events (via git calls?): [watcher] node event executing refresh. When this happens, it looks like children exploration and watch registration happens to directories which so far have been leaves themselves. That way, nvim-tree gradually continues digging folder hierarchy (and adding watchers) without folders even been expanded.

My questions/suggestions:

  1. If I expand a folder, only immediate children of this folder should be monitored at worst.
  2. If a folder is collapsed, it would be nice to stop watching its children recursively (e.g. if children were expanded as well). I see that with expanded children this might be tricky.
  3. I know there is an option to exclude (blacklist) certain folders from watching. But it would extremely helpful in this scenario to be able to do the opposite: watch only included folders (whitelist). That is, watching is disabled systemwide so I can use nvim-tree for general file system browsing without being afraid it starts choking with file system watch events, but my projects directory is white-listed, so it's monitored once I start browsing it.
Neovim version
NVIM v0.11.6
Build type: Debug
LuaJIT 2.1.1741730670
Operating system and version

Windows 11 Pro

Windows variant

No response

nvim-tree version

e16cd38

Clean room replication
vim.g.loaded_netrw = 1
vim.g.loaded_netrwPlugin = 1

vim.cmd([[set runtimepath=$VIMRUNTIME]])
vim.cmd([[set packpath=/tmp/nvt-min/site]])
local package_root = "/tmp/nvt-min/site/pack"
local install_path = package_root .. "/packer/start/packer.nvim"
local function load_plugins()
  require("packer").startup({
    {
      "wbthomason/packer.nvim",
      "nvim-tree/nvim-tree.lua",
      "nvim-tree/nvim-web-devicons",
      -- ADD PLUGINS THAT ARE _NECESSARY_ FOR REPRODUCING THE ISSUE
    },
    config = {
      package_root = package_root,
      compile_path = install_path .. "/plugin/packer_compiled.lua",
      display = { non_interactive = true },
    },
  })
end
if vim.fn.isdirectory(install_path) == 0 then
  print("Installing nvim-tree and dependencies.")
  vim.fn.system({ "git", "clone", "--depth=1", "https://github.com/wbthomason/packer.nvim", install_path })
end
load_plugins()
require("packer").sync()
vim.cmd([[autocmd User PackerComplete ++once echo "Ready!" | lua setup()]])
vim.opt.termguicolors = true
vim.opt.cursorline = true

-- MODIFY NVIM-TREE SETTINGS THAT ARE _NECESSARY_ FOR REPRODUCING THE ISSUE
_G.setup = function()
---@type nvim_tree.config
  local config = {
    disable_netrw = true,
    hijack_cursor = true,
    sync_root_with_cwd = true,
    update_focused_file = {
        enable = true,
        update_root = false,
    }
  }
  require("nvim-tree").setup(config)
end

-- UNCOMMENT this block for diagnostics issues, substituting pattern and cmd as appropriate.
-- Requires diagnostics.enable = true in setup.
--[[
vim.api.nvim_create_autocmd("FileType", {
  pattern = "lua",
  callback = function()
    vim.lsp.start {
      name = "my-luals",
      cmd = { "lua-language-server" },
      root_dir = vim.loop.cwd(),
    }
  end,
})
]]
Steps to reproduce
  1. c:;cd c:\
  2. nvim.exe -nu path-to\nvt-min.lua
  3. :NvimTreeOpen
  4. Start expanding the tree: open Users, then your user home folder. No need to go deeper.
  5. Possibly do some other activities on the system to trigger file system events (e.g. build something in VS, browse with the browser etc.)
Expected behavior

I expect nvim-tree not ro consume tons of memory just because of expanding couple of folders.

Actual behavior

Nvim-tree consumes huge amount of memory which keeps growing indefinitely.
Image

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the watcher registration and refresh paths in lua/nvim-tree/explorer/init.lua and lua/nvim-tree/explorer/watch.lua, then reproduce on Windows using the provided clean-room setup and expansion steps. Trace why watching grows beyond expanded folders and why callbacks accumulate. Done means expanding a few folders no longer causes indefinitely growing memory or watcher registrations.

Written by the indexing model from the issue text.

Assessment

Tech stack
lua
Domain
devtools, operating-systems
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.