airbytehq / airbytehq/airbyte

Destination Typesense: increase timeout and fix batch size

Offen
#24,097 7 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
area/connectors community connectors/destination/typesense frozen releaseStage/alpha team/destinations type/bug
Vorherrschende Sprache
Python
Sterne
22.1k
Forks
5.3k
PR-Merge-Kennzahlen
PR-Kennzahlen ausstehend

Beschreibung

## Environment
- **Airbyte version**: 0.42.0
- **OS Version / Instance**: Windows 10
- **Deployment**: Docker
- **Source Connector and version**: PostgreSQL 1.0.48
- **Destination Connector and version**: Typesense 0.1.0
- **Step where error happened**: Sync job

## Current Behavior
When syncing a large dataset to Typesense the connector times out due to a low timeout config in the client's `connection_timeout_seconds` param. Currently it is set to 2 seconds in the destination.py file and should be at least 1 hour.

Also, the batch size is set to `"type": "string",` in the `spec.json` file when it should be `"type": "integer"`.

## Expected Behavior
Connector should not time out even when syncing large amounts of data.

## Logs
urllib3.exceptions.ProtocolError: ('Connection aborted.', timeout('The write operation timed out'))

## Steps to Reproduce
1. Set up an airbyte connection with typesense as destination
2. Try to sync large amounts of data (millions of records)

Beitragsleitfaden

Beitragsleitfaden öffnen

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.