Document recommendations for block size and page size given an expected number of writers

I sent a mail on dev list but I seem to have a problem with email on dev list so opening a bug here .

I am on a multithreaded system where there are M threads , each thread creating an independent parquet writer and writing on the hdfs in its own independent files . I have a finite amount of RAM say R .

Now when I created parquet writer using default block and page size i get heap error (no memory ) on my set up . so I reduced my block size and page size to very low and my system stopped giving me these out of memory errors and started writing the file correctly . I am able to read these files correctly as well .

I should not have to make the memory low and parquet should automatically make sure i do not get these errors .

But in case i have to keep track of the memory my question is as follows.

Now keeping these values very less is not a recommended practice as i would loose on performance . I am particularly concerned about write performance . What math formula do you recommend that I should use to find correct blockSize , pageSize to be passed to the parquet constructor to have the right WRITE performance . ie how can i decide what should be the right blockSize , pageSize for a parquet writer given that i have M threads and total RAM memory available is R . I don't understand dictionaryPageSize need and in case i need to bother about that as well kindly let me know but i have kept enableDictionary flag as false .

I am using the bellow constructor .
public More ...ParquetWriter(
162 Path file,
163 WriteSupport writeSupport,
164 CompressionCodecName compressionCodecName,
165 int blockSize,
166 int pageSize,
167 int dictionaryPageSize,
168 boolean enableDictionary,
169 boolean validating,
170 WriterVersion writerVersion,
171 Configuration conf) throws IOException {

**Reporter**: [Manish Agarwal](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=manish.agarwal)

_{**Note**: *This issue was originally created as [PARQUET-156](https://issues.apache.org/jira/browse/PARQUET-156). Please see the [migration documentation](https://issues.apache.org/jira/browse/PARQUET-2502) for further details.*}

ParquetWriter コンストラクタと、その blockSize、pageSize、dictionaryPageSize、enableDictionary パラメータから始めます。M 個の writer と総メモリ R について、プロジェクトがサポートできる指針を明らかにし、そのうえでトレードオフと dictionary page に関する考慮事項を文書化します。完了条件は、ユーザーが書き込みパフォーマンスに重点を置いた、実行可能なサイズ設定の推奨事項を得られることです。

索引モデルが issue の本文から書いたものです。