Benchmarking the cost of reading/writing large state objects

The problem

In #2196 this comment was made

Should we change the datamodel to be the same as the new one in the forum? why or why not? it would certainly reduce the computational load of cleanup up a proposal discussion.

As it currently stands, the data model for the forum we store each post within the thread structure, meaning, that assuming a size of 40 bytes for each post(the hash of the post + author id) this mean a thread of 3000 posts could take ~120KB of storage.

The problem with this approach is that an stored thread could get too big and make the reads/writes to storage really slow. This is not accounted for in the weight functions since they only count number of read/writes and multiply by the average time for a read/write.

Just taking into account the SSD sequential read speeds of 3500 MB/s that would be ~34,286,000 units of weights(picoseconds). And the RocksDbWeight for a read in the runtime is 25,000,000 this means that at this size, it might very well affect the weight function significantly.

However, as explained here turns out that there is too much variability(a lot of outliers) when trying to measure the performance of the database for a given record size. (Note: that video talks about number of operation vs time but a conversation with substrate developers the same problem, where the variance was too great to create a correlation function, existed between size and time)

We note that the variance for a record read/write is not a problem since it can't be used as a vector for an attack. The attacker can't control the conditions for the outliers so when abusing from an extrinsic with this variance after multiple calls they are going to have the average time, meaning it won't be exploited.

And this is why Substrate use the average read/write time for a kusama-like distribution for the RocksDbWeight. But we know that in their case the size of the records are actually constrained and we might have much larger ones and affect the average weight of a read/write by a much larger margin.

Proposed solution

To deal with this we propose the following:

• Benchmark the database by adapting the existing db benchmarks into a size vs time benchmark.
• We expect to find a correlation between the average read/write time and a record size. However, to find an statistically significant variation the record size might need to be way bigger than the sizes that concern us
• If that's not the case we can add a factor to the relevant extrinsic's weight functions to account for the added weight given the size of the stored record and use the weight refund to prevent from always charging the worst-case scenario
• If given the max size of the record this is unacceptable, add an extrinsic parameter for the size of the record which will check along the other parameters of the extrinsic. To check for the size of a thread without actually reading the thread we need to store in a map the length of a thread in a separate map, to prevent adding another invariance this approach is not preferred if it can be prevented.

In this issue I will document the benchmarking process.

┆Issue is synchronized with this Asana task by Unito

Start with the existing database benchmarks linked in the issue and the runtime's RocksDbWeight and relevant extrinsic weight functions. Document a size-versus-time benchmarking process, report whether record size affects average read/write costs, and identify which proposed weighting approach the results support.