Destination Typesense: increase timeout and fix batch size
- Vorherrschende Sprache
- Python
- Sterne
- 22.1k
- Forks
- 5.3k
- PR-Merge-Kennzahlen
- PR-Kennzahlen ausstehend
Beschreibung
## Environment
- **Airbyte version**: 0.42.0
- **OS Version / Instance**: Windows 10
- **Deployment**: Docker
- **Source Connector and version**: PostgreSQL 1.0.48
- **Destination Connector and version**: Typesense 0.1.0
- **Step where error happened**: Sync job
## Current Behavior
When syncing a large dataset to Typesense the connector times out due to a low timeout config in the client's `connection_timeout_seconds` param. Currently it is set to 2 seconds in the destination.py file and should be at least 1 hour.
Also, the batch size is set to `"type": "string",` in the `spec.json` file when it should be `"type": "integer"`.
## Expected Behavior
Connector should not time out even when syncing large amounts of data.
## Logs
urllib3.exceptions.ProtocolError: ('Connection aborted.', timeout('The write operation timed out'))
## Steps to Reproduce
1. Set up an airbyte connection with typesense as destination
2. Try to sync large amounts of data (millions of records)
Beitragsleitfaden
Bewertung
Dieses Issue wurde noch nicht bewertet.