Self-Hosted Write Limits

Description

Hello,

I have a few questions. I've deployed Typesense to 3 Google Compute instances (2vCPUs & 8GB RAM) with a load balancer. My current setup works great for serving quick search results but fails when upserting 100k records in a queue with HTTP 503 Lagging or Not Ready. I was wondering if there are any write limits to the self-hosted version or if it is just based on CPU limitations. If it’s based on CPU, what do you suggest setting the --healthy-write-lag flag in the server config to? Is this value equal to writes per second or is it queue-based? My system triggers an update per record individually so I'm using upsert over import. Importing in batches is probably the better solution, is this also affected by --healthy-write-lag or would I run into the same issue? Another solution is increasing the number of cores but more costly. I'm also going to try and reduce my indexed fields.

Each instance has 20GB SSDs which are almost at capacity with only 4 collections with 20k documents each. My next question is where is all this storage being depleted from? Should I run a scheduled cleanup of logs, state, or some other directory?

Any help is appreciated.
Thanks

Start with the self-hosted deployment on Google Compute instances and the --healthy-write-lag setting, then examine the reported HTTP 503 during individual upserts and batch imports. Determine how write readiness and storage usage behave, including whether logs or state consume the SSD; done means documenting the limits, configuration semantics, and cleanup guidance.