Indexers require a lot of memory to avoid OOM kill

When indexing the GH archive dataset, instances like c5a.2xlarge with 16 GB RAM are insufficient. The OOM killer repetitively killed each of my instances during merge operations.
Everything works with a c5a.4xlarge.

My indexing settings are

• commit timeout: 60 secs
• target num split docs: 10 million

Here is a screenshot where we can easily spot when the pod quickwit-indexer-0 was killed at 12:50 (time starts at 12:40 and ends at 13h).

And here are the logs from quickwit-indexer-0

2023-07-13T12:51:11.725Z  INFO quickwit_indexing::actors::indexer: send-to-index-serializer commit_trigger=Timeout split_ids=01H57MTHJZBAJ9JRFQK8XPGB04 num_docs=605413
2023-07-13T12:51:11.763Z  INFO quickwit_indexing::actors::indexer: new-split split_id=01H57MWC7ET8V3YXZC738BNMY6 partition_id=0
2023-07-13T12:51:48.411Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57MTHJYGKT73MBKBDEWHPX4}:packager: quickwit_indexing::actors::packager: start-packaging-splits split_ids=["01H57MTHJZBAJ9JRFQK8XPGB04"]
2023-07-13T12:51:48.411Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57MTHJYGKT73MBKBDEWHPX4}:packager: quickwit_indexing::actors::packager: create-packaged-split split_id="01H57MTHJZBAJ9JRFQK8XPGB04"
2023-07-13T12:51:48.434Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57MTHJYGKT73MBKBDEWHPX4}:uploader: quickwit_indexing::actors::uploader: start-stage-and-store-splits split_ids=["01H57MTHJZBAJ9JRFQK8XPGB04"]
2023-07-13T12:51:56.912Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57MTHJYGKT73MBKBDEWHPX4}:uploader:upload{split=01H57MTHJZBAJ9JRFQK8XPGB04}:store_split: quickwit_indexing::split_store::indexing_split_store: store-split-remote-success split_size_in_megabytes=917.63556 num_docs=605413 elapsed_secs=8.277697 throughput_mb_s=110.85639 is_mature=false
2023-07-13T12:51:56.912Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57MTHJYGKT73MBKBDEWHPX4}:uploader:upload{split=01H57MTHJZBAJ9JRFQK8XPGB04}:store_split: quickwit_indexing::split_store::indexing_split_store: store-in-cache
2023-07-13T12:51:57.288Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57MTHJYGKT73MBKBDEWHPX4}:publisher{split_update=SplitsUpdate { index_id: "gh-archive", new_splits: "01H57MTHJZBAJ9JRFQK8XPGB04", checkpoint_delta: Some(gh-archive-source:∆(https://data.gharchive.org/2022-09-29-0.json.gz:(00000000000085121040..18446744073709551615] https://data.gharchive.org/2022-09-29-12.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-09-30-0.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-09-30-12.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-01-0.json.gz:(00000000000000000000..00000000000205358762])) }}: quickwit_indexing::actors::publisher: publish-new-splits new_splits=["01H57MTHJZBAJ9JRFQK8XPGB04"] checkpoint_delta=Some(gh-archive-source:∆(https://data.gharchive.org/2022-09-29-0.json.gz:(00000000000085121040..18446744073709551615] https://data.gharchive.org/2022-09-29-12.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-09-30-0.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-09-30-12.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-01-0.json.gz:(00000000000000000000..00000000000205358762]))
2023-07-13T12:51:57.288Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57MTHJYGKT73MBKBDEWHPX4}:publisher{split_update=SplitsUpdate { index_id: "gh-archive", new_splits: "01H57MTHJZBAJ9JRFQK8XPGB04", checkpoint_delta: Some(gh-archive-source:∆(https://data.gharchive.org/2022-09-29-0.json.gz:(00000000000085121040..18446744073709551615] https://data.gharchive.org/2022-09-29-12.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-09-30-0.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-09-30-12.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-01-0.json.gz:(00000000000000000000..00000000000205358762])) }}: quickwit_metastore::checkpoint: delta=∆(https://data.gharchive.org/2022-09-29-0.json.gz:(00000000000085121040..18446744073709551615] https://data.gharchive.org/2022-09-29-12.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-09-30-0.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-09-30-12.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-01-0.json.gz:(00000000000000000000..00000000000205358762]) checkpoint=Ckpt()
2023-07-13T12:51:57.289Z  INFO quickwit_indexing::actors::merge_planner: Planned merge operation. merge_operation=Merge(operation_type=Merge, merged_split_id=01H57MXRQ9H34TDX74PCHCTH1A,splits=[01H567V9DKRMA79ZMX5NTMFV2N,01H566ZHGZZYW9RJ4MCWBV7M70,])
2023-07-13T12:52:11.812Z  INFO quickwit_indexing::actors::indexer: send-to-index-serializer commit_trigger=Timeout split_ids=01H57MWC7ET8V3YXZC738BNMY6 num_docs=691992
2023-07-13T12:52:11.844Z  INFO quickwit_indexing::actors::indexer: new-split split_id=01H57MY6X5696GZQEJMHM1CAY2 partition_id=0
2023-07-13T12:52:44.848Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57MWC7DH19GAWKYDBZ6WC6Z}:packager: quickwit_indexing::actors::packager: start-packaging-splits split_ids=["01H57MWC7ET8V3YXZC738BNMY6"]
2023-07-13T12:52:44.848Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57MWC7DH19GAWKYDBZ6WC6Z}:packager: quickwit_indexing::actors::packager: create-packaged-split split_id="01H57MWC7ET8V3YXZC738BNMY6"
2023-07-13T12:52:44.874Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57MWC7DH19GAWKYDBZ6WC6Z}:uploader: quickwit_indexing::actors::uploader: start-stage-and-store-splits split_ids=["01H57MWC7ET8V3YXZC738BNMY6"]
2023-07-13T12:52:53.183Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57MWC7DH19GAWKYDBZ6WC6Z}:uploader:upload{split=01H57MWC7ET8V3YXZC738BNMY6}:store_split: quickwit_indexing::split_store::indexing_split_store: store-split-remote-success split_size_in_megabytes=872.7403 num_docs=691992 elapsed_secs=8.069917 throughput_mb_s=108.14737 is_mature=false
2023-07-13T12:52:53.183Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57MWC7DH19GAWKYDBZ6WC6Z}:uploader:upload{split=01H57MWC7ET8V3YXZC738BNMY6}:store_split: quickwit_indexing::split_store::indexing_split_store: store-in-cache
2023-07-13T12:52:53.453Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57MWC7DH19GAWKYDBZ6WC6Z}:publisher{split_update=SplitsUpdate { index_id: "gh-archive", new_splits: "01H57MWC7ET8V3YXZC738BNMY6", checkpoint_delta: Some(gh-archive-source:∆(https://data.gharchive.org/2022-10-01-0.json.gz:(00000000000205358762..18446744073709551615] https://data.gharchive.org/2022-10-01-12.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-02-0.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-02-12.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-03-0.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-03-12.json.gz:(00000000000000000000..00000000000200432142])) }}: quickwit_indexing::actors::publisher: publish-new-splits new_splits=["01H57MWC7ET8V3YXZC738BNMY6"] checkpoint_delta=Some(gh-archive-source:∆(https://data.gharchive.org/2022-10-01-0.json.gz:(00000000000205358762..18446744073709551615] https://data.gharchive.org/2022-10-01-12.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-02-0.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-02-12.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-03-0.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-03-12.json.gz:(00000000000000000000..00000000000200432142]))
2023-07-13T12:52:53.453Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57MWC7DH19GAWKYDBZ6WC6Z}:publisher{split_update=SplitsUpdate { index_id: "gh-archive", new_splits: "01H57MWC7ET8V3YXZC738BNMY6", checkpoint_delta: Some(gh-archive-source:∆(https://data.gharchive.org/2022-10-01-0.json.gz:(00000000000205358762..18446744073709551615] https://data.gharchive.org/2022-10-01-12.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-02-0.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-02-12.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-03-0.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-03-12.json.gz:(00000000000000000000..00000000000200432142])) }}: quickwit_metastore::checkpoint: delta=∆(https://data.gharchive.org/2022-10-01-0.json.gz:(00000000000205358762..18446744073709551615] https://data.gharchive.org/2022-10-01-12.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-02-0.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-02-12.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-03-0.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-03-12.json.gz:(00000000000000000000..00000000000200432142]) checkpoint=Ckpt()
2023-07-13T12:52:53.453Z  INFO quickwit_indexing::actors::merge_planner: Planned merge operation. merge_operation=Merge(operation_type=Merge, merged_split_id=01H57MZFJDKMGN5G33BXS22XT0,splits=[01H56X84YR3H2E965S1VN8QZ4J,01H568CHZVMHAG5H3M7P5F3F4G,])
2023-07-13T12:53:11.864Z  INFO quickwit_indexing::actors::indexer: send-to-index-serializer commit_trigger=Timeout split_ids=01H57MY6X5696GZQEJMHM1CAY2 num_docs=536572
2023-07-13T12:53:11.894Z  INFO quickwit_indexing::actors::indexer: new-split split_id=01H57N01HSKYG12Y18CNJSZ0TA partition_id=0
2023-07-13T12:53:44.621Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57MY6X44TBRWBZFYE3GJ9DQ}:packager: quickwit_indexing::actors::packager: start-packaging-splits split_ids=["01H57MY6X5696GZQEJMHM1CAY2"]
2023-07-13T12:53:44.621Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57MY6X44TBRWBZFYE3GJ9DQ}:packager: quickwit_indexing::actors::packager: create-packaged-split split_id="01H57MY6X5696GZQEJMHM1CAY2"
2023-07-13T12:53:44.637Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57MY6X44TBRWBZFYE3GJ9DQ}:uploader: quickwit_indexing::actors::uploader: start-stage-and-store-splits split_ids=["01H57MY6X5696GZQEJMHM1CAY2"]
2023-07-13T12:53:52.404Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57MY6X44TBRWBZFYE3GJ9DQ}:uploader:upload{split=01H57MY6X5696GZQEJMHM1CAY2}:store_split: quickwit_indexing::split_store::indexing_split_store: store-split-remote-success split_size_in_megabytes=844.13763 num_docs=536572 elapsed_secs=7.578652 throughput_mb_s=111.38361 is_mature=false
2023-07-13T12:53:52.404Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57MY6X44TBRWBZFYE3GJ9DQ}:uploader:upload{split=01H57MY6X5696GZQEJMHM1CAY2}:store_split: quickwit_indexing::split_store::indexing_split_store: store-in-cache
2023-07-13T12:53:52.633Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57MY6X44TBRWBZFYE3GJ9DQ}:publisher{split_update=SplitsUpdate { index_id: "gh-archive", new_splits: "01H57MY6X5696GZQEJMHM1CAY2", checkpoint_delta: Some(gh-archive-source:∆(https://data.gharchive.org/2022-10-03-12.json.gz:(00000000000200432142..18446744073709551615] https://data.gharchive.org/2022-10-04-0.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-04-12.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-05-0.json.gz:(00000000000000000000..00000000000566074882])) }}: quickwit_indexing::actors::publisher: publish-new-splits new_splits=["01H57MY6X5696GZQEJMHM1CAY2"] checkpoint_delta=Some(gh-archive-source:∆(https://data.gharchive.org/2022-10-03-12.json.gz:(00000000000200432142..18446744073709551615] https://data.gharchive.org/2022-10-04-0.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-04-12.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-05-0.json.gz:(00000000000000000000..00000000000566074882]))
2023-07-13T12:53:52.633Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57MY6X44TBRWBZFYE3GJ9DQ}:publisher{split_update=SplitsUpdate { index_id: "gh-archive", new_splits: "01H57MY6X5696GZQEJMHM1CAY2", checkpoint_delta: Some(gh-archive-source:∆(https://data.gharchive.org/2022-10-03-12.json.gz:(00000000000200432142..18446744073709551615] https://data.gharchive.org/2022-10-04-0.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-04-12.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-05-0.json.gz:(00000000000000000000..00000000000566074882])) }}: quickwit_metastore::checkpoint: delta=∆(https://data.gharchive.org/2022-10-03-12.json.gz:(00000000000200432142..18446744073709551615] https://data.gharchive.org/2022-10-04-0.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-04-12.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-05-0.json.gz:(00000000000000000000..00000000000566074882]) checkpoint=Ckpt()
2023-07-13T12:54:11.887Z  INFO quickwit_indexing::actors::indexer: send-to-index-serializer commit_trigger=Timeout split_ids=01H57N01HSKYG12Y18CNJSZ0TA num_docs=512558
2023-07-13T12:54:12.110Z  INFO quickwit_indexing::actors::indexer: new-split split_id=01H57N1W5GVKXNG8J2BNN10D61 partition_id=0
2023-07-13T12:54:50.789Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57N01HRYJH2VXHJ0XFGB1BB}:packager: quickwit_indexing::actors::packager: start-packaging-splits split_ids=["01H57N01HSKYG12Y18CNJSZ0TA"]
2023-07-13T12:54:50.789Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57N01HRYJH2VXHJ0XFGB1BB}:packager: quickwit_indexing::actors::packager: create-packaged-split split_id="01H57N01HSKYG12Y18CNJSZ0TA"
2023-07-13T12:54:50.863Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57N01HRYJH2VXHJ0XFGB1BB}:uploader: quickwit_indexing::actors::uploader: start-stage-and-store-splits split_ids=["01H57N01HSKYG12Y18CNJSZ0TA"]
2023-07-13T12:54:58.886Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57N01HRYJH2VXHJ0XFGB1BB}:uploader:upload{split=01H57N01HSKYG12Y18CNJSZ0TA}:store_split: quickwit_indexing::split_store::indexing_split_store: store-split-remote-success split_size_in_megabytes=867.86194 num_docs=512558 elapsed_secs=7.8917985 throughput_mb_s=109.97011 is_mature=false
2023-07-13T12:54:58.886Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57N01HRYJH2VXHJ0XFGB1BB}:uploader:upload{split=01H57N01HSKYG12Y18CNJSZ0TA}:store_split: quickwit_indexing::split_store::indexing_split_store: store-in-cache
2023-07-13T12:54:59.019Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57N01HRYJH2VXHJ0XFGB1BB}:publisher{split_update=SplitsUpdate { index_id: "gh-archive", new_splits: "01H57N01HSKYG12Y18CNJSZ0TA", checkpoint_delta: Some(gh-archive-source:∆(https://data.gharchive.org/2022-10-05-0.json.gz:(00000000000566074882..18446744073709551615] https://data.gharchive.org/2022-10-05-12.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-06-0.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-06-12.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-07-0.json.gz:(00000000000000000000..00000000000445783296])) }}: quickwit_indexing::actors::publisher: publish-new-splits new_splits=["01H57N01HSKYG12Y18CNJSZ0TA"] checkpoint_delta=Some(gh-archive-source:∆(https://data.gharchive.org/2022-10-05-0.json.gz:(00000000000566074882..18446744073709551615] https://data.gharchive.org/2022-10-05-12.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-06-0.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-06-12.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-07-0.json.gz:(00000000000000000000..00000000000445783296]))
2023-07-13T12:54:59.019Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57N01HRYJH2VXHJ0XFGB1BB}:publisher{split_update=SplitsUpdate { index_id: "gh-archive", new_splits: "01H57N01HSKYG12Y18CNJSZ0TA", checkpoint_delta: Some(gh-archive-source:∆(https://data.gharchive.org/2022-10-05-0.json.gz:(00000000000566074882..18446744073709551615] https://data.gharchive.org/2022-10-05-12.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-06-0.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-06-12.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-07-0.json.gz:(00000000000000000000..00000000000445783296])) }}: quickwit_metastore::checkpoint: delta=∆(https://data.gharchive.org/2022-10-05-0.json.gz:(00000000000566074882..18446744073709551615] https://data.gharchive.org/2022-10-05-12.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-06-0.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-06-12.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-07-0.json.gz:(00000000000000000000..00000000000445783296]) checkpoint=Ckpt()
2023-07-13T12:55:12.093Z  INFO quickwit_indexing::actors::indexer: send-to-index-serializer commit_trigger=Timeout split_ids=01H57N1W5GVKXNG8J2BNN10D61 num_docs=747623
2023-07-13T12:55:12.126Z  INFO quickwit_indexing::actors::indexer: new-split split_id=01H57N3PYY3HTKWEB3MVKVE62T partition_id=0
2023-07-13T12:55:47.040Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57N1W5FQVV1D0YBWS1BJ41V}:packager: quickwit_indexing::actors::packager: start-packaging-splits split_ids=["01H57N1W5GVKXNG8J2BNN10D61"]
2023-07-13T12:55:47.040Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57N1W5FQVV1D0YBWS1BJ41V}:packager: quickwit_indexing::actors::packager: create-packaged-split split_id="01H57N1W5GVKXNG8J2BNN10D61"
2023-07-13T12:55:47.059Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57N1W5FQVV1D0YBWS1BJ41V}:uploader: quickwit_indexing::actors::uploader: start-stage-and-store-splits split_ids=["01H57N1W5GVKXNG8J2BNN10D61"]
2023-07-13T12:55:55.173Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57N1W5FQVV1D0YBWS1BJ41V}:uploader:upload{split=01H57N1W5GVKXNG8J2BNN10D61}:store_split: quickwit_indexing::split_store::indexing_split_store: store-split-remote-success split_size_in_megabytes=887.82104 num_docs=747623 elapsed_secs=7.9114027 throughput_mb_s=112.220436 is_mature=false
2023-07-13T12:55:55.173Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57N1W5FQVV1D0YBWS1BJ41V}:uploader:upload{split=01H57N1W5GVKXNG8J2BNN10D61}:store_split: quickwit_indexing::split_store::indexing_split_store: store-in-cache
2023-07-13T12:56:12.132Z  INFO quickwit_indexing::actors::indexer: send-to-index-serializer commit_trigger=Timeout split_ids=01H57N3PYY3HTKWEB3MVKVE62T num_docs=588910
2023-07-13T12:56:12.386Z  INFO quickwit_indexing::actors::indexer: new-split split_id=01H57N5HK5GC36H3RTW16YJ793 partition_id=0
2023-07-13T12:56:35.127Z  INFO merge{merge_split_id=01H57MXRQ9H34TDX74PCHCTH1A split_ids=["01H567V9DKRMA79ZMX5NTMFV2N", "01H566ZHGZZYW9RJ4MCWBV7M70"] typ=Merge}:merge_split_downloader: quickwit_indexing::actors::merge_split_downloader: download-merge-splits dir=/quickwit/qwdata/indexing/gh-archive%01H55B319XCP8E97Q2CP1RA90D%gh-archive-source%0%BT3pxN/merge%5XQO2B
2023-07-13T12:56:35.128Z  INFO quickwit_indexing::actors::merge_planner: Planned merge operation. merge_operation=Merge(operation_type=Merge, merged_split_id=01H57N681RCT1ZEEHKAERYB1QS,splits=[01H56ZHEZNAY0Q4BH42MMKHRC7,01H56YA65XJ9ZMDJ0RMYZ6A0BV,])
2023-07-13T12:56:35.333Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57N1W5FQVV1D0YBWS1BJ41V}:publisher{split_update=SplitsUpdate { index_id: "gh-archive", new_splits: "01H57N1W5GVKXNG8J2BNN10D61", checkpoint_delta: Some(gh-archive-source:∆(https://data.gharchive.org/2022-10-07-0.json.gz:(00000000000445783296..18446744073709551615] https://data.gharchive.org/2022-10-07-12.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-08-0.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-08-12.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-09-0.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-09-12.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-10-0.json.gz:(00000000000000000000..00000000000015052639])) }}: quickwit_indexing::actors::publisher: publish-new-splits new_splits=["01H57N1W5GVKXNG8J2BNN10D61"] checkpoint_delta=Some(gh-archive-source:∆(https://data.gharchive.org/2022-10-07-0.json.gz:(00000000000445783296..18446744073709551615] https://data.gharchive.org/2022-10-07-12.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-08-0.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-08-12.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-09-0.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-09-12.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-10-0.json.gz:(00000000000000000000..00000000000015052639]))
2023-07-13T12:56:35.334Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57N1W5FQVV1D0YBWS1BJ41V}:publisher{split_update=SplitsUpdate { index_id: "gh-archive", new_splits: "01H57N1W5GVKXNG8J2BNN10D61", checkpoint_delta: Some(gh-archive-source:∆(https://data.gharchive.org/2022-10-07-0.json.gz:(00000000000445783296..18446744073709551615] https://data.gharchive.org/2022-10-07-12.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-08-0.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-08-12.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-09-0.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-09-12.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-10-0.json.gz:(00000000000000000000..00000000000015052639])) }}: quickwit_metastore::checkpoint: delta=∆(https://data.gharchive.org/2022-10-07-0.json.gz:(00000000000445783296..18446744073709551615] https://data.gharchive.org/2022-10-07-12.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-08-0.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-08-12.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-09-0.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-09-12.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-10-0.json.gz:(00000000000000000000..00000000000015052639]) checkpoint=Ckpt()
2023-07-13T12:56:44.934Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57N3PYXQFDQD3RZK1R01AQK}:packager: quickwit_indexing::actors::packager: start-packaging-splits split_ids=["01H57N3PYY3HTKWEB3MVKVE62T"]
2023-07-13T12:56:44.934Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57N3PYXQFDQD3RZK1R01AQK}:packager: quickwit_indexing::actors::packager: create-packaged-split split_id="01H57N3PYY3HTKWEB3MVKVE62T"
2023-07-13T12:56:44.950Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57N3PYXQFDQD3RZK1R01AQK}:uploader: quickwit_indexing::actors::uploader: start-stage-and-store-splits split_ids=["01H57N3PYY3HTKWEB3MVKVE62T"]
2023-07-13T12:56:52.902Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57N3PYXQFDQD3RZK1R01AQK}:uploader:upload{split=01H57N3PYY3HTKWEB3MVKVE62T}:store_split: quickwit_indexing::split_store::indexing_split_store: store-split-remote-success split_size_in_megabytes=840.8641 num_docs=588910 elapsed_secs=7.73065 throughput_mb_s=108.770164 is_mature=false
2023-07-13T12:56:52.902Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57N3PYXQFDQD3RZK1R01AQK}:uploader:upload{split=01H57N3PYY3HTKWEB3MVKVE62T}:store_split: quickwit_indexing::split_store::indexing_split_store: store-in-cache
2023-07-13T12:57:12.204Z  INFO quickwit_indexing::actors::indexer: send-to-index-serializer commit_trigger=Timeout split_ids=01H57N5HK5GC36H3RTW16YJ793 num_docs=535515
2023-07-13T12:57:12.531Z  INFO quickwit_indexing::actors::indexer: new-split split_id=01H57N7C8ES2T0JKJ5RGYC1TVZ partition_id=0
2023-07-13T12:57:49.396Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57N5HK4Q1XV0D59G5CVP1CX}:packager: quickwit_indexing::actors::packager: start-packaging-splits split_ids=["01H57N5HK5GC36H3RTW16YJ793"]
2023-07-13T12:57:49.396Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57N5HK4Q1XV0D59G5CVP1CX}:packager: quickwit_indexing::actors::packager: create-packaged-split split_id="01H57N5HK5GC36H3RTW16YJ793"
2023-07-13T12:57:49.421Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57N5HK4Q1XV0D59G5CVP1CX}:uploader: quickwit_indexing::actors::uploader: start-stage-and-store-splits split_ids=["01H57N5HK5GC36H3RTW16YJ793"]
2023-07-13T12:57:57.206Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57N5HK4Q1XV0D59G5CVP1CX}:uploader:upload{split=01H57N5HK5GC36H3RTW16YJ793}:store_split: quickwit_indexing::split_store::indexing_split_store: store-split-remote-success split_size_in_megabytes=797.1218 num_docs=535515 elapsed_secs=7.598126 throughput_mb_s=104.910324 is_mature=false
2023-07-13T12:57:57.206Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57N5HK4Q1XV0D59G5CVP1CX}:uploader:upload{split=01H57N5HK5GC36H3RTW16YJ793}:store_split: quickwit_indexing::split_store::indexing_split_store: store-in-cache
2023-07-13T12:58:12.290Z  INFO quickwit_indexing::actors::indexer: send-to-index-serializer commit_trigger=Timeout split_ids=01H57N7C8ES2T0JKJ5RGYC1TVZ num_docs=650124
2023-07-13T12:58:12.325Z  INFO quickwit_indexing::actors::indexer: new-split split_id=01H57N96Y3D6G8P2DGVKT6S7A4 partition_id=0
2023-07-13T12:58:44.603Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57N7C8C1NFMPF5GEVPJSJ3M}:packager: quickwit_indexing::actors::packager: start-packaging-splits split_ids=["01H57N7C8ES2T0JKJ5RGYC1TVZ"]
2023-07-13T12:58:44.604Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57N7C8C1NFMPF5GEVPJSJ3M}:packager: quickwit_indexing::actors::packager: create-packaged-split split_id="01H57N7C8ES2T0JKJ5RGYC1TVZ"
2023-07-13T12:58:44.631Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57N7C8C1NFMPF5GEVPJSJ3M}:uploader: quickwit_indexing::actors::uploader: start-stage-and-store-splits split_ids=["01H57N7C8ES2T0JKJ5RGYC1TVZ"]
2023-07-13T12:59:12.312Z  INFO quickwit_indexing::actors::indexer: send-to-index-serializer commit_trigger=Timeout split_ids=01H57N96Y3D6G8P2DGVKT6S7A4 num_docs=622383
2023-07-13T12:59:12.705Z  INFO quickwit_indexing::actors::indexer: new-split split_id=01H57NB1HSHMKS2S2S4CCKQ26Z partition_id=0
2023-07-13T12:59:28.928Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57N7C8C1NFMPF5GEVPJSJ3M}:uploader:upload{split=01H57N7C8ES2T0JKJ5RGYC1TVZ}:store_split: quickwit_indexing::split_store::indexing_split_store: store-split-remote-success split_size_in_megabytes=818.33435 num_docs=650124 elapsed_secs=44.122208 throughput_mb_s=18.546995 is_mature=false
2023-07-13T12:59:28.929Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57N7C8C1NFMPF5GEVPJSJ3M}:uploader:upload{split=01H57N7C8ES2T0JKJ5RGYC1TVZ}:store_split: quickwit_indexing::split_store::indexing_split_store: store-in-cache
2023-07-13T12:59:47.243Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57N96Y2D8CY3TKV8F53QF6M}:packager: quickwit_indexing::actors::packager: start-packaging-splits split_ids=["01H57N96Y3D6G8P2DGVKT6S7A4"]
2023-07-13T12:59:47.243Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57N96Y2D8CY3TKV8F53QF6M}:packager: quickwit_indexing::actors::packager: create-packaged-split split_id="01H57N96Y3D6G8P2DGVKT6S7A4"
2023-07-13T12:59:47.272Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57N96Y2D8CY3TKV8F53QF6M}:uploader: quickwit_indexing::actors::uploader: start-stage-and-store-splits split_ids=["01H57N96Y3D6G8P2DGVKT6S7A4"]
2023-07-13T12:59:55.673Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57N96Y2D8CY3TKV8F53QF6M}:uploader:upload{split=01H57N96Y3D6G8P2DGVKT6S7A4}:store_split: quickwit_indexing::split_store::indexing_split_store: store-split-remote-success split_size_in_megabytes=827.62897 num_docs=622383 elapsed_secs=8.104421 throughput_mb_s=102.12068 is_mature=false
2023-07-13T12:59:55.673Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57N96Y2D8CY3TKV8F53QF6M}:uploader:upload{split=01H57N96Y3D6G8P2DGVKT6S7A4}:store_split: quickwit_indexing::split_store::indexing_split_store: store-in-cache
2023-07-13T13:00:12.404Z  INFO quickwit_indexing::actors::indexer: send-to-index-serializer commit_trigger=Timeout split_ids=01H57NB1HSHMKS2S2S4CCKQ26Z num_docs=555336
2023-07-13T13:00:12.441Z  INFO quickwit_indexing::actors::indexer: new-split split_id=01H57NCW7N13ZN54W0NFQ3DZ8S partition_id=0
2023-07-13T13:00:48.281Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57NB1HREWXFQ055SB7AKF63}:packager: quickwit_indexing::actors::packager: start-packaging-splits split_ids=["01H57NB1HSHMKS2S2S4CCKQ26Z"]
2023-07-13T13:00:48.281Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57NB1HREWXFQ055SB7AKF63}:packager: quickwit_indexing::actors::packager: create-packaged-split split_id="01H57NB1HSHMKS2S2S4CCKQ26Z"
2023-07-13T13:00:58.515Z  INFO merge{merge_split_id=01H57MZFJDKMGN5G33BXS22XT0 split_ids=["01H56X84YR3H2E965S1VN8QZ4J", "01H568CHZVMHAG5H3M7P5F3F4G"] typ=Merge}:merge_split_downloader: quickwit_indexing::actors::merge_split_downloader: download-merge-splits dir=/quickwit/qwdata/indexing/gh-archive%01H55B319XCP8E97Q2CP1RA90D%gh-archive-source%0%BT3pxN/merge%AF3hzl
2023-07-13T13:00:58.516Z  INFO quickwit_indexing::actors::merge_planner: Planned merge operation. merge_operation=Merge(operation_type=Merge, merged_split_id=01H57NE98MQ4F4TSYBXGG55WM3,splits=[01H57N01HSKYG12Y18CNJSZ0TA,01H57MY6X5696GZQEJMHM1CAY2,01H57MWC7ET8V3YXZC738BNMY6,01H57MTHJZBAJ9JRFQK8XPGB04,01H57KYJED4MM01K5JCDEX43JW,01H57KWQK05B9D06PAWN2T45S6,01H57KTWVCP59AXS03TN20J2CG,01H57KS26P0S4T707ZD9G1WZKC,01H57KQ7K34DYPQXBQ3PBQR6J4,01H57KNCXX0MHCWADTM9199SP7,])
2023-07-13T13:00:58.516Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57NB1HREWXFQ055SB7AKF63}:uploader: quickwit_indexing::actors::uploader: start-stage-and-store-splits split_ids=["01H57NB1HSHMKS2S2S4CCKQ26Z"]
2023-07-13T13:00:58.969Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57N3PYXQFDQD3RZK1R01AQK}:publisher{split_update=SplitsUpdate { index_id: "gh-archive", new_splits: "01H57N3PYY3HTKWEB3MVKVE62T", checkpoint_delta: Some(gh-archive-source:∆(https://data.gharchive.org/2022-10-10-0.json.gz:(00000000000015052639..18446744073709551615] https://data.gharchive.org/2022-10-10-12.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-11-0.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-11-12.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-12-0.json.gz:(00000000000000000000..00000000000115210514])) }}: quickwit_indexing::actors::publisher: publish-new-splits new_splits=["01H57N3PYY3HTKWEB3MVKVE62T"] checkpoint_delta=Some(gh-archive-source:∆(https://data.gharchive.org/2022-10-10-0.json.gz:(00000000000015052639..18446744073709551615] https://data.gharchive.org/2022-10-10-12.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-11-0.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-11-12.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-12-0.json.gz:(00000000000000000000..00000000000115210514]))
2023-07-13T13:00:58.969Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57N3PYXQFDQD3RZK1R01AQK}:publisher{split_update=SplitsUpdate { index_id: "gh-archive", new_splits: "01H57N3PYY3HTKWEB3MVKVE62T", checkpoint_delta: Some(gh-archive-source:∆(https://data.gharchive.org/2022-10-10-0.json.gz:(00000000000015052639..18446744073709551615] https://data.gharchive.org/2022-10-10-12.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-11-0.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-11-12.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-12-0.json.gz:(00000000000000000000..00000000000115210514])) }}: quickwit_metastore::checkpoint: delta=∆(https://data.gharchive.org/2022-10-10-0.json.gz:(00000000000015052639..18446744073709551615] https://data.gharchive.org/2022-10-10-12.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-11-0.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-11-12.json.gz:(00000000000000000000..18446744073709551615] https://data.gharchive.org/2022-10-12-0.json.gz:(00000000000000000000..00000000000115210514]) checkpoint=Ckpt()
2023-07-13T13:01:07.453Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57NB1HREWXFQ055SB7AKF63}:uploader:upload{split=01H57NB1HSHMKS2S2S4CCKQ26Z}:store_split: quickwit_indexing::split_store::indexing_split_store: store-split-remote-success split_size_in_megabytes=842.98895 num_docs=555336 elapsed_secs=8.729204 throughput_mb_s=96.57111 is_mature=false
2023-07-13T13:01:07.453Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57NB1HREWXFQ055SB7AKF63}:uploader:upload{split=01H57NB1HSHMKS2S2S4CCKQ26Z}:store_split: quickwit_indexing::split_store::indexing_split_store: store-in-cache
2023-07-13T13:01:12.681Z  INFO quickwit_indexing::actors::indexer: send-to-index-serializer commit_trigger=Timeout split_ids=01H57NCW7N13ZN54W0NFQ3DZ8S num_docs=567101
2023-07-13T13:01:12.711Z  INFO quickwit_indexing::actors::indexer: new-split split_id=01H57NEQ3A6NZEECW9C074A3HK partition_id=0
2023-07-13T13:01:48.441Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57NCW7MB8FWZMMB1VZZHNN6}:packager: quickwit_indexing::actors::packager: start-packaging-splits split_ids=["01H57NCW7N13ZN54W0NFQ3DZ8S"]
2023-07-13T13:01:48.442Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57NCW7MB8FWZMMB1VZZHNN6}:packager: quickwit_indexing::actors::packager: create-packaged-split split_id="01H57NCW7N13ZN54W0NFQ3DZ8S"
2023-07-13T13:02:12.709Z  INFO quickwit_indexing::actors::indexer: send-to-index-serializer commit_trigger=Timeout split_ids=01H57NEQ3A6NZEECW9C074A3HK num_docs=650166
2023-07-13T13:02:12.854Z  INFO quickwit_indexing::actors::indexer: new-split split_id=01H57NGHQ61JG2GJQSKVQP2C37 partition_id=0
2023-07-13T13:02:46.584Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57NEQ39PVM6Q50EJ5G834SC}:packager: quickwit_indexing::actors::packager: start-packaging-splits split_ids=["01H57NEQ3A6NZEECW9C074A3HK"]
2023-07-13T13:02:46.585Z  INFO index-doc-batches{index_id=gh-archive source_id=gh-archive-source pipeline_ord=0 workbench_id=01H57NEQ39PVM6Q50EJ5G834SC}:packager: quickwit_indexing::actors::packager: create-packaged-split split_id="01H57NEQ3A6NZEECW9C074A3HK"
2023-07-13T13:03:12.732Z  INFO quickwit_indexing::actors::indexer: send-to-index-serializer commit_trigger=Timeout split_ids=01H57NGHQ61JG2GJQSKVQP2C37 num_docs=582765
2023-07-13T13:03:13.180Z  INFO quickwit_indexing::actors::indexer: new-split split_id=01H57NJCAXKXNVHVX1FD28E0PG partition_id=0
2023-07-13T13:04:12.816Z  INFO quickwit_indexing::actors::indexer: send-to-index-serializer commit_trigger=Timeout split_ids=01H57NJCAXKXNVHVX1FD28E0PG num_docs=571749
2023-07-13T13:04:12.847Z  INFO quickwit_indexing::actors::indexer: new-split split_id=01H57NM70H208C4P6WXGXEF3CS partition_id=0
2023-07-13T13:04:15.278Z  INFO quickwit_cluster::change: Node `quickwit-indexer-2` has joined the cluster. cluster_id=gharchive-demo-quickwit node_id=quickwit-indexer-2
2023-07-13T13:04:16.278Z  INFO quickwit_cluster::change: Node `quickwit-indexer-2` has transitioned to ready state. cluster_id=gharchive-demo-quickwit node_id=quickwit-indexer-2
2023-07-13T13:04:17.277Z  INFO quickwit_cluster::change: Node `quickwit-indexer-2` has left the cluster. cluster_id=gharchive-demo-quickwit node_id=quickwit-indexer-2
2023-07-13T13:05:12.873Z  INFO quickwit_indexing::actors::indexer: send-to-index-serializer commit_trigger=Timeout split_ids=01H57NM70H208C4P6WXGXEF3CS num_docs=543629
rpc error: code = NotFound desc = an error occurred when try to find container "ab4f957b087b20407aa0e8b35e0d20d686d921724b043c0b851849302010beae": not found%