quickwit-oss / quickwit-oss/quickwit
exit_status=downstream actor exited
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 11.7k
- Forks
- 597
- Avg merge
- 2d 22h
- Merged PRs (30d)
- 37
Description
Describe the bug
A clear and concise description of what the bug is.
I'm not very familiar with quickwit. Initially, the log ingestion worked fine, but after a while, I noticed a large number of errors. The data storage is on MinIO, and the indexing is on Postgres. The deployment was done using Helm and Terraform. The error messages are as follows:
2025-08-22T16:42:23.013Z INFO quickwit_actors::spawn_builder: actor-exit actor_id=quickwit_indexing::actors::index_serializer::IndexSerializer-twilight-tj9q exit_status=success
2025-08-22T16:42:23.013Z INFO quickwit_actors::spawn_builder: no more messages actor="Packager-withered-4LMj"
2025-08-22T16:42:23.013Z INFO quickwit_actors::spawn_builder: actor-exit actor_id=Packager-withered-4LMj exit_status=success
2025-08-22T16:42:23.013Z INFO quickwit_actors::spawn_builder: no more messages actor="IndexUploader-hidden-7bjg"
2025-08-22T16:42:23.013Z INFO quickwit_actors::spawn_builder: actor-exit actor_id=IndexUploader-hidden-7bjg exit_status=success
2025-08-22T16:42:23.013Z INFO quickwit_actors::spawn_builder: no more messages actor="quickwit_indexing::actors::sequencer::Sequencer<quickwit_indexing::actors::publisher::Publisher>-falling-n9hF"
2025-08-22T16:42:23.013Z INFO quickwit_actors::spawn_builder: actor-exit actor_id=quickwit_indexing::actors::sequencer::Sequencer<quickwit_indexing::actors::publisher::Publisher>-falling-n9hF exit_status=success
2025-08-22T16:42:23.013Z INFO quickwit_actors::spawn_builder: no more messages actor="Publisher-spring-zHGO"
2025-08-22T16:42:23.013Z INFO quickwit_actors::spawn_builder: actor-exit actor_id=Publisher-spring-zHGO exit_status=success
2025-08-22T16:42:23.022Z INFO quickwit_actors::spawn_builder: actor-exit actor_id=quickwit_indexing::actors::doc_processor::DocProcessor-shy-AgX2 exit_status=downstream actor exited
2025-08-22T16:42:23.022Z ERROR quickwit_actors::actor_context: exit activating-kill-switch actor=quickwit_indexing::actors::doc_processor::DocProcessor-shy-AgX2 exit_status=DownstreamClosed
2025-08-22T16:42:23.023Z INFO quickwit_actors::spawn_builder: actor-exit actor_id=SourceActor-withered-UTbm exit_status=killed
2025-08-22T16:42:23.437Z ERROR quickwit_actors::actor_handle: actor-exit-without-success actor="SourceActor-withered-UTbm"
2025-08-22T16:42:23.437Z ERROR quickwit_actors::actor_handle: actor-exit-without-success actor="quickwit_indexing::actors::doc_processor::DocProcessor-shy-AgX2"
2025-08-22T16:42:23.437Z ERROR quickwit_indexing::actors::indexing_pipeline: Indexing pipeline failure. pipeline_id=IndexingPipelineId { node_id: "vt-quickwit-indexer-0", index_uid: IndexUid { index_id: "files", incarnation_id: Ulid(2121395130011908575915230040062802881) }, source_id: "_ingest-api-source", pipeline_uid: Pipeline(01K2973MTRGM5EXAFVFVA4KD1X) } generation=522937 healthy_actors=["Indexer-crimson-j5QY"] failed_or_unhealthy_actors=["SourceActor-withered-UTbm", "quickwit_indexing::actors::doc_processor::DocProcessor-shy-AgX2"] success_actors=["quickwit_indexing::actors::index_serializer::IndexSerializer-twilight-tj9q", "Packager-withered-4LMj", "IndexUploader-hidden-7bjg", "quickwit_indexing::actors::sequencer::Sequencer<quickwit_indexing::actors::publisher::Publisher>-falling-n9hF", "Publisher-spring-zHGO"]
here is the config:
{
name = "config.default_index_root_uri"
value = "s3://quickwit/indexes"
},
{
name = "config.metastore_uri"
value = "postgres://${local.postgres_user}:quickwitStrong12%23@vt-quickwit-minio-postgresql/${local.postgres_db}"
},
{
#
name = "config.storage.s3.endpoint"
value = "http://4.5.0.3:9000"
},
{
name = "config.storage.s3.flavor"
value = "minio"
},
{
name = "config.storage.s3.access_key_id"
value = local.access_key_id
},
{
name = "config.storage.s3.force_path_style_access"
value = true
},
{
name = "config.storage.s3.secret_access_key"
value = local.secret_access_key
},
{
name = "config.storage.s3.region"
value = "us-east-1"
},
{
name = "ingress.enabled"
value = true
},
{
name = "ingress.ingressClassName"
value = "nginx"
},
{
name = "ingress.hostname"
value = "quickwit.local"
},
{
name = "config.indexer.split_store_max_num_bytes"
value = "200G"
},
{
name = "indexer.replicaCount"
value = 3
},
{
name = "config.indexer.split_store_max_num_splits"
value = "10000"
},
{
name = "config.ingest_api.max_queue_memory_usage"
value = "20GiB"
},
{
name = "config.ingest_api.max_queue_disk_usage"
value = "30GiB"
},
{
name = "config.searcher.fast_field_cache_capacity"
value = "10G"
},
{
name = "config.searcher.split_footer_cache_capacity"
value = "10G"
},
{
name = "config.searcher.max_num_concurrent_split_streams"
value = "100"
}
- Output of
quickwit --version
0.8.1 - The index_config.yaml
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the supplied indexing-pipeline and actor-exit logs, focusing on SourceActor, DocProcessor, and the downstream actor chain. Then verify the Quickwit 0.8.1 configuration for the Postgres metastore and S3 storage in the Helm/Terraform deployment. Done means identifying the cause of the shutdown and documenting a reproducible fix or the missing diagnostic data.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- helm, postgres, rust, terraform
- Domain
- backend, cloud, databases, devops, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100