Controlling memory utilization by ParquetReader

In Apache Druid, Parquet is one of the popular form of input source to ingest data into a druid cluster (https://druid.apache.org/docs/latest/development/extensions-core/parquet.html). We rely on the parquet-mr library to read the parquet files and then convert them into Druid native format row-for-row to ingest. A considerable amount of our usecases ingest the whole parquet files (ie all columns in a single shot) into the system.

A challenge that we face is that the parquet reader loads up an entire row group into memory as part of its normal operation. Row groups can be quite large (like, 1GB large) and sometimes it creates a pressure on our reader JVM leading to OOMs. Further, in some other cases it ends up creating GC pressure on the JVM leading to a decrease in the throughput of the ingestion tasks.

To mitigate this problem, we are considering that would it be better to have an option to download the Parquet rowgroup/file first and memory-map it for reading? The code which buffers the rowgroup works on the ByteBuffer interface already (https://github.com/apache/parquet-mr/blob/master/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/ParquetFileReader.java#L1763), so it seems like it could compliment the MMappedByteBuffer implementation too. Such a thing would alleviate pressure off of our reader JVM there by heavily reducing the chances for OOMs.

We're very open to more ideas or already tried solutions around this problem. 

**Reporter**: [Rohan Garg](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=rohangarg)

_{**Note**: *This issue was originally created as [PARQUET-2141](https://issues.apache.org/jira/browse/PARQUET-2141). Please see the [migration documentation](https://issues.apache.org/jira/browse/PARQUET-2502) for further details.*}

Commencez par parquet-hadoop/src/main/java/org/apache/parquet/hadoop/ParquetFileReader.java autour de la ligne 1763 et examinez comment la mise en tampon des groupes de lignes utilise ByteBuffer. Évaluez l’approche MMappedByteBuffer proposée par rapport au comportement actuel, puis définissez et validez une solution convenue qui réduise la mémoire utilisée par la JVM et la pression du GC sans modifier les résultats de l’ingestion.

Rédigé par le modèle d'indexation à partir du texte de l'issue.