Buggy statistics reporting in stable and v2.0.2?

Hi,
I'm using mimalloc in my linux (ubuntu18.04.5lts) application and am investigating why my RSS keeps growing. This is proving difficult because I see negative values in the "current" allocation column.
I've changed to v2.0.2 (and forced MI_DEBUG=2) and still see the problem.
My options are as follows:

mimalloc: option 'show_errors': 1
mimalloc: option 'show_stats': 1
mimalloc: option 'eager_commit': 1
mimalloc: option 'eager_region_commit': 1
mimalloc: option 'reset_decommits': 1
mimalloc: option 'large_os_pages': 0
mimalloc: option 'reserve_huge_os_pages': 0
mimalloc: option 'reserve_os_memory': 0
mimalloc: option 'segment_cache': 0
mimalloc: option 'page_reset': 1
mimalloc: option 'abandoned_page_reset': 1
mimalloc: option 'segment_reset': 1
mimalloc: option 'eager_commit_delay': 1
mimalloc: option 'allow_decommit': 1
mimalloc: option 'reset_delay': 1
mimalloc: option 'segment_decommit_delay': 1
mimalloc: option 'use_numa_nodes': 0
mimalloc: option 'limit_os_alloc': 0
mimalloc: option 'os_tag': 100
mimalloc: option 'max_errors': 1600
mimalloc: option 'max_warnings': 1600
mimalloc: process init: 0x7ffff7fcf740
mimalloc: debug level : 2
mimalloc: secure level: 0

Is that a reasonable set? The delays are very short as I want to correlate the RSS frees back to my application.

With my reproducer I see a negative commit-current, and the commit-free is almost double the commit-total:

Allocating 1000000 words
mimalloc: using 2 numa regions
heap stats:     peak      total      freed    current       unit      count  
normal   6:     192 b      1.1 kb     1.1 kb       0 b       48 b       25     ok
normal   9:     320 b      800 b      800 b        0 b       80 b       10     ok
normal  23:     896 b      896 b      896 b        0 b      896 b        1     ok
normal  37:    20.0 kb    50.1 kb    50.1 kb       0 b     10.0 kb       5     ok

heap stats:     peak      total      freed    current       unit      count  
    normal:    20.5 k     52.7 k     52.7 k        0        1.2 kb      41     ok
     large:       0 b        0 b        0 b        0 b        1 b              ok
      huge:     7.7 m      7.7 m      7.7 m        0        7.7 mb       1     ok
     total:     7.7 mb     7.8 mb     7.8 mb       0 b        1 b              ok
malloc req:     7.6 mb     7.6 mb     7.6 mb       0 b        1 b              ok

  reserved:    16.0 mb    16.0 mb     8.0 mb     8.0 mb       1 b              not all freed!
 committed:     8.2 mb     8.3 mb    15.8 mb    -7.4 mb       1 b              ok
     reset:       0 b        0 b        0 b        0 b        1 b              ok
   touched:     7.8 mb     7.8 mb     7.8 mb       0 b        1 b              ok
  segments:       2          2          2          0                           ok
-abandoned:       0          0          0          0                           ok
   -cached:       0          0          0          0                           ok
     pages:       4          5          5          0                           ok
-abandoned:       0          0          0          0                           ok
 -extended:       5   
 -noretire:      26   
     mmaps:       2   
   commits:       3   
   threads:       0          0          0          0                           ok
  searches:     0.0 avg
numa nodes:       2
   elapsed:       4.016 s
   process: user: 0.021 s, system: 0.009 s, faults: 0, rss: 36.2 mb, commit: 8.2 mb
mimalloc: process done: 0x7fa265941740

My initial debugging suggests that my large buffer is freed via mi_free_block_mt() for the actual size freed, but this does not update the commit mask. Then mi_segment_os_free() frees with a full (~0) commit mask of memory, leading tothe double counting.

So, are my flags valid, and if so should mt_free_block_mt() update the commit mask?

My repro code is

#include <mimalloc.h>
#include <mimalloc-new-delete.h>
#include <iostream>
#include <vector>

int main(int argc, char **argv) {
  if (argc<2) {
    std::cerr<<"Usage stats_minimal size\n";
    return 1;
  }
  auto n = std::strtoull(argv[1], nullptr, 0);
  std::cerr<<"Allocating " << n << " words\n";
  std::vector<std::size_t> v(n);
  for (std::size_t i{0}; i != n; ++i)
    v[i] = i;
  std::cerr<<"Allocating " << n << " words\n";
  mi_stats_print(NULL);
  v.resize(0);
  v.shrink_to_fit(); // This is the point at which the "current commit" value becomes negative.
  return 0;
}

compiled with

g++ -g --std=c++17 -Wall -I ~/work/install/mimalloc/include/mimalloc-2.0 -fno-builtin-malloc -fno-builtin-calloc -fno-builtin-realloc -fno-builtin-free stats_minimal.cpp  ~/work/build/poplar/deb/install/mimalloc/lib/mimalloc-2.0/libmimalloc.a -lpthread

and I'm invoking the test via

./a.out 1000000 

Inizia compilando ed eseguendo il reproducer fornito stats_minimal.cpp con il comando e le opzioni mostrati. Segui il percorso segnalato attraverso mi_free_block_mt() e mi_segment_os_free(), concentrandoti su come viene contabilizzata la commit mask durante vector shrink_to_fit(). Il lavoro è completato quando il reproducer non segnala più un current commit negativo né un commit-free maggiore di commit-total.