Debug assert fails in routing_filter_add(), "(index_no / addrs_per_page < pages_per_extent)": Causes large inserts workload to fail.

This specific problem was encountered during benchmarking of single-client inserting 20M rows (in PG-SplinterDB integration code base).

Update (agurajada; 4/2023) After deeper investigations of variations of this repro, the basic issue seems to be that there is some instability in the library in /main when using small key-value pairs. After some experimentation, it appears that we can stably insert 20+M rows, using single / multiple threads, when key=4 bytes and value >= 20 bytes. For smaller k/v pair sizes, there are different forms of instabilities seen. We really need a comprehensive test-suite that can exercise these basic insert workloads for diff # of rows inserts, # of clients, and varying combinations of K/V pair sizes.

This has been reproduced using standalone test script off of /main @ SHA 89f09b3.

Branch: agurajada/560-rf-add-bug (Has another commit, which fixes another issue, partially tracked by the failures by repros attempted for issue #458. You need that fix in order to go further along to repro this failure.)

The failure is seen in this test large_inserts_stress_test --num-inserts 20000000 test_560_seq_htobe32_key_random_6byte_values_inserts from this branch:

Running 1 CTests, suite name 'large_inserts_stress', test case 'test_560_seq_htobe32_key_random_6byte_values_inserts'.
TEST 1/1 large_inserts_stress:test_seq_htobe32_key_random_6byte_values_inserts OS-pid=426343, OS-tid=426343, Thread-ID=0, Assertion failed at src/routing_filter.c:586:routing_filter_add(): "(index_no / addrs_per_page < pages_per_extent)". index_no=16384, addrs_per_page=512, (index_no / addrs_per_page)=32, pages_per_extent=32
Aborted (core dumped)

With release build, you will get a seg-fault a few lines later.

The test case test_560_seq_htobe32_key_random_6byte_values_inserts() has the exact conditions required to trigger this bug. There are other test cases in this test file that invoke different combinations of sequential / random key inserts, and they all seem to succeed.