Changes to write throughput in version 6 releases with a write-friendly configuration

This isn't a bug but it was confusing to me so I will document it here while I figure out whether there is anything to be done. I ran tools/benchmark.sh, then increased the options that tolerate compaction debt (# files in L0, soft/hard pending compaction bytes limits) and repeated a shorter sequence (fillseq + overwrite). With this change that I call a write-friendly (mis)configuration the throughput drops significantly in version 6.3, then improves in 6.13 and is stable through 6.26 although the average throughput in 6.26 is still worse than it was prior to 6.3. Overwrites/s were ~150k prior to 6.3, drop to ~80k in 6.3, then increase to 110k in 6.13.

The throughput drop in 6.3 is from [3d9d77d](https://github.com/facebook/rocksdb/commit/3d9d77d) which makes intra-L0 compactions respect the max_compaction_bytes option. This is probably something I requested -- see [issue 6889](https://github.com/facebook/rocksdb/issues/6889), somewhat related are [issue 7996](https://github.com/facebook/rocksdb/issues/7996) and [issue 4991](https://github.com/facebook/rocksdb/issues/4991). Note that max_compaction_bytes is 0 by default, and when 0 the [value is adjusted](https://github.com/facebook/rocksdb/blob/55a2105258b63d17879db005391dea81c53e3212/db/column_family.cc#L364) to 25 * target_file_size_base

The increase in 6.13 is from [177f8bd](https://github.com/facebook/rocksdb/commit/177f8bd). This adjusts the L0 target size used to compute the L0 compaction score. AFAIK,.this reduces the compaction score in some cases when there are write-heavy workloads this leading to more throughput.

Throughput from the overwrite test:
```
# ops_sec - operations per second
# mb_sec - ops_sec * size-of-operation-in-MB
# db_size - database size
# c_wgb - GB written by compaction
# w_amp - Write-amplification as (bytes written by compaction / bytes written by memtable flush)
# c_mbps - Average write rate for compaction
# c_secs - Wall clock seconds doing compaction
# usec_op - Microseconds per operation
# p50, p99, p99.9, p99.99 - 50th, 99th, 99.9th, 99.99th percentile response time in usecs
# pmax - max response time in usecs
# uptime - RocksDB uptime in seconds
# stall% - Percentage of time writes are stalled
# Nstall - Number of stalls
# u_cpu - #seconds/1000 of user CPU
# s_cpu - #seconds/1000 of system CPU
# test - Name of test
# date - Date/time of test
# version - RocksDB version
# job_id - User-provided job ID
ops_sec mb_sec db_size c_wgb w_amp c_mbps c_secs usec_op p50 p99 p99.9 p99.99 pmax uptime stall% Nstall u_cpu s_cpu test date version job_id
155060 62.1 421GB 785.0 7.0 448.52 2712.8 103.2 21.5 1266 3736 7331 531008 1792 73.9 7909 8.9 1.9 overwrite.t16.s0 2022-01-7T13:26:04 6.0
145861 58.4 415GB 790.8 7.5 450.62 2695.3 109.7 22.7 1265 3512 9343 1345074 1797 73.9 7897 8.9 1.9 overwrite.t16.s0 2022-01-7T14:13:14 6.1
169700 68.0 431GB 825.5 6.7 471.24 2802.1 94.3 21.9 1257 3811 9115 609141 1794 70.7 7636 9.8 1.9 overwrite.t16.s0 2022-01-7T15:00:15 6.2
80615 32.3 372GB 689.5 12.3 395.93 2257.4 198.5 23.0 3014 5566 8822 554721 1783 85.7 4759 5.5 1.5 overwrite.t16.s0 2022-01-7T15:47:12 6.3
81519 32.7 374GB 662.2 11.3 379.54 2228.5 196.2 21.5 2932 5284 8155 65387 1787 85.4 4786 5.5 1.5 overwrite.t16.s0 2022-01-7T16:34:08 6.4
76484 30.6 371GB 650.9 11.8 370.96 2136.1 209.1 22.5 3046 5408 8462 35361 1797 86.0 4709 5.3 1.5 overwrite.t16.s0 2022-01-7T17:21:14 6.5
77726 31.1 372GB 668.4 11.9 380.06 2219.8 205.8 21.5 2923 5054 7504 45054 1801 86.4 4700 5.2 1.5 overwrite.t16.s0 2022-01-7T18:08:01 6.6
78322 31.4 372GB 666.9 11.8 382.14 2214.4 204.3 21.1 2959 5357 8312 350091 1787 86.4 4873 5.2 1.5 overwrite.t16.s0 2022-01-7T18:54:41 6.7
77072 30.9 372GB 654.2 11.8 373.08 2193.1 207.5 22.0 2904 5097 7770 65085 1796 86.3 4610 5.3 1.5 overwrite.t16.s0 2022-01-7T19:41:46 6.8
80035 32.1 373GB 660.4 11.5 377.65 2273.0 199.9 21.6 2997 5428 8342 46092 1791 85.9 4671 5.4 1.4 overwrite.t16.s0 2022-01-7T20:28:55 6.10
74788 30.0 369GB 649.2 12.1 371.93 2188.2 213.9 23.3 2994 5572 8697 501113 1787 86.0 4646 5.3 1.4 overwrite.t16.s0 2022-01-7T21:18:31 6.11
75538 30.3 370GB 634.5 11.6 360.98 2142.6 211.7 24.1 3110 5493 8287 328947 1800 85.4 4754 5.5 1.4 overwrite.t16.s0 2022-01-7T22:08:16 6.12
116114 46.5 395GB 833.9 10.0 474.70 3032.1 137.8 23.1 1279 4503 13723 1717833 1799 76.4 18742 7.8 1.7 overwrite.t16.s0 2022-01-7T22:57:49 6.13
115778 46.4 138.2 22.5 1286 5023 18959 1266024 1796 75.2 7.6 1.6 overwrite.t16.s0 2022-01-7T23:51:38 6.14
110037 44.1 393GB 753.7 9.5 429.12 2816.6 145.4 23.0 1292 5569 19089 889392 1798 75.8 11592 7.5 1.6 overwrite.t16.s0 2022-01-8T00:45:38 6.15
108450 43.4 392GB 704.9 9.1 406.02 2570.3 147.5 22.3 1432 6404 19149 610891 1778 76.6 9584 7.3 1.6 overwrite.t16.s0 2022-01-8T01:39:49 6.16
110603 44.3 394GB 729.4 9.1 415.76 2655.9 144.6 23.1 1293 4541 18036 160949 1796 75.8 9383 7.5 1.6 overwrite.t16.s0 2022-01-8T02:34:15 6.17
110509 44.3 392GB 704.7 9.0 403.27 2767.2 144.8 23.4 2664 6311 19684 1177615 1790 74.0 11578 7.7 1.5 overwrite.t16.s0 2022-01-8T03:28:35 6.19
106794 42.8 391GB 707.0 9.2 403.34 2741.8 149.8 23.1 2681 6316 18437 1554163 1795 74.8 11620 7.5 1.5 overwrite.t16.s0 2022-01-8T04:23:04 6.20
110646 44.3 395GB 695.5 8.7 397.30 2687.1 144.6 23.1 2672 5936 17141 2101904 1793 73.9 8409 7.6 1.5 overwrite.t16.s0 2022-01-8T05:17:24 6.22
109930 44.0 393GB 668.2 8.5 384.02 2629.5 145.5 23.2 2686 6009 16802 183276 1782 73.9 7326 7.6 1.5 overwrite.t16.s0 2022-01-8T06:11:45 6.23
109221 43.7 393GB 678.1 8.7 390.00 2669.0 146.5 22.9 2690 6148 16862 163933 1780 74.0 7281 7.6 1.5 overwrite.t16.s0 2022-01-8T07:06:28 6.24
108191 43.3 391GB 692.0 9.0 396.32 2707.7 147.9 23.2 2686 6240 18433 748524 1788 74.3 9322 7.6 1.5 overwrite.t16.s0 2022-01-8T08:00:56 6.25
109060 43.7 393GB 701.2 8.9 399.14 2605.5 146.7 23.4 2684 5811 17181 181751 1799 73.9 7907 7.5 1.5 overwrite.t16.s0 2022-01-8T08:55:48 6.26
```

Command lines, first run fillseq, then run overwrite

Fillseq
```
/usr/bin/time -f '%e %U %S' -o bm.lc.nt16.cm1.d0/v6.0.2/benchmark_fillseq.wal_disabled.v400.log.time numactl --interleave=all ./db_bench --benchmarks=fillseq --allow_concurrent_memtable_write=false --level0_file_num_compaction_trigger=4 --level0_slowdown_writes_trigger=40 --level0_stop_writes_trigger=60 --max_background_jobs=8 --max_write_buffer_number=8 --db=/data/m/rx --wal_dir=/data/m/rx --num=800000000 --num_levels=8 --key_size=20 --value_size=400 --block_size=8192 --cache_size=51539607552 --cache_numshardbits=6 --compression_max_dict_bytes=0 --compression_ratio=0.5 --compression_type=none --bytes_per_sync=8388608 --cache_index_and_filter_blocks=1 --cache_high_pri_pool_ratio=0.5 --benchmark_write_rate_limit=0 --write_buffer_size=16777216 --target_file_size_base=16777216 --max_bytes_for_level_base=67108864 --verify_checksum=1 --delete_obsolete_files_period_micros=62914560 --max_bytes_for_level_multiplier=8 --statistics=0 --stats_per_interval=1 --stats_interval_seconds=20 --histogram=1 --memtablerep=skip_list --bloom_bits=10 --open_files=-1 --subcompactions=1 --compaction_style=0 --min_level_to_compress=3 --level_compaction_dynamic_level_bytes=true --pin_l0_filter_and_index_blocks_in_cache=1 --soft_pending_compaction_bytes_limit=1680405954560 --hard_pending_compaction_bytes_limit=3360811909120 --min_level_to_compress=0 --use_existing_db=0 --sync=0 --threads=1 --memtablerep=vector --allow_concurrent_memtable_write=false --disable_wal=1 --seed=1641589769
```

Overwrite
```
/usr/bin/time -f '%e %U %S' -o bm.lc.nt16.cm1.d0/v6.0.2/benchmark_overwrite.t16.s0.log.time numactl --interleave=all ./db_bench --benchmarks=overwrite --use_existing_db=1 --sync=0 --level0_file_num_compaction_trigger=4 --level0_slowdown_writes_trigger=40 --level0_stop_writes_trigger=60 --max_background_jobs=8 --max_write_buffer_number=8 --db=/data/m/rx --wal_dir=/data/m/rx --num=800000000 --num_levels=8 --key_size=20 --value_size=400 --block_size=8192 --cache_size=51539607552 --cache_numshardbits=6 --compression_max_dict_bytes=0 --compression_ratio=0.5 --compression_type=none --bytes_per_sync=8388608 --cache_index_and_filter_blocks=1 --cache_high_pri_pool_ratio=0.5 --benchmark_write_rate_limit=0 --write_buffer_size=16777216 --target_file_size_base=16777216 --max_bytes_for_level_base=67108864 --verify_checksum=1 --delete_obsolete_files_period_micros=62914560 --max_bytes_for_level_multiplier=8 --statistics=0 --stats_per_interval=1 --stats_interval_seconds=20 --histogram=1 --memtablerep=skip_list --bloom_bits=10 --open_files=-1 --subcompactions=1 --compaction_style=0 --min_level_to_compress=3 --level_compaction_dynamic_level_bytes=true --pin_l0_filter_and_index_blocks_in_cache=1 --soft_pending_compaction_bytes_limit=1680405954560 --hard_pending_compaction_bytes_limit=3360811909120 --duration=1800 --threads=16 --merge_operator="put" --seed=1641590763
```