airbytehq / airbytehq/PyAirbyte

Airtable connector fails

Abierto
#275 1 comentario 3 reacciones 1 asignado Reclamado por @aaronsteers Ver en GitHub
Lenguaje dominante
Python
Estrellas
344
Forks
77
Merge medio
1 d 11 h
PR fusionados (30 d)
35

Descripción

When reading from a successfully connected airtable connector, the connector looks for a file on the local cache, no file exists and the sync fails.

I tried to simplify the example as much as possible:

```
# Import PyAirbyte
import airbyte as ab

# Create and install airtable connector
source: ab.Source = ab.get_source("source-airtable")

# Configure the source
source.set_config(
config={
"credentials": {
"auth_method": "api_key",
"api_key": "MyKey"
},
},
)
# Verify the config and creds by running `check`:
source.check()

source.select_streams("*")

read_result: ab.ReadResult = source.read()

# Print the read result
print("Read result:")
print(read_result)
```

Error:

```
FileNotFoundError: [Errno 2] No such file or directory: '.cache/default_cache/talentclass/all_leads/tblnpRi7u0YECVout_01J01NPY9AKJ84BQKKK6S91BNR.jsonl.gz'
```

Full logs:

```
(myenv) enrique@Enrique-TUF:~/pyAirbyteTests$ python airtableTests.py

Connection check succeeded for `source-airtable`.
Started `source-airtable` read operation at 20:13:59...
Failed `source-airtable` read operation at 20:14:04.
Traceback (most recent call last):
File "/home/enrique/pyAirbyteTests/myenv/lib/python3.10/site-packages/airbyte/sources/base.py", line 734, in read
cache_processor.process_airbyte_messages(
File "/home/enrique/pyAirbyteTests/myenv/lib/python3.10/site-packages/airbyte/_future_cdk/record_processor.py", line 196, in process_airbyte_messages
self.process_record_message(
File "/home/enrique/pyAirbyteTests/myenv/lib/python3.10/site-packages/airbyte/_future_cdk/sql_processor.py", line 237, in process_record_message
self.file_writer.process_record_message(
File "/home/enrique/pyAirbyteTests/myenv/lib/python3.10/site-packages/airbyte/_processors/file/base.py", line 155, in process_record_message
batch_handle = self._new_batch(stream_name=stream_name)
File "/home/enrique/pyAirbyteTests/myenv/lib/python3.10/site-packages/airbyte/_processors/file/base.py", line 106, in _new_batch
batch_handle = BatchHandle(
File "/home/enrique/pyAirbyteTests/myenv/lib/python3.10/site-packages/airbyte/_batch_handles.py", line 30, in __init__
self._open_file_writer: IO[bytes] = file_opener(self._files[0])
File "/home/enrique/pyAirbyteTests/myenv/lib/python3.10/site-packages/airbyte/_processors/file/jsonl.py", line 34, in _open_new_file
return cast(IO[bytes], gzip.open(file_path, "w"))
File "/usr/lib/python3.10/gzip.py", line 58, in open
binary_file = GzipFile(filename, gz_mode, compresslevel)
File "/usr/lib/python3.10/gzip.py", line 174, in __init__
fileobj = self.myfileobj = builtins.open(filename, mode or 'rb')
FileNotFoundError: [Errno 2] No such file or directory: '.cache/default_cache/talentclass/all_leads/tblnpRi7u0YECVout_01J01NPY9AKJ84BQKKK6S91BNR.jsonl.gz'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/enrique/pyAirbyteTests/airtableTests.py", line 23, in
read_result: ab.ReadResult = source.read()
File "/home/enrique/pyAirbyteTests/myenv/lib/python3.10/site-packages/airbyte/sources/base.py", line 745, in read
raise exc.AirbyteConnectorFailedError(
airbyte.exceptions.AirbyteConnectorFailedError: AirbyteConnectorFailedError: Connector failed.

Log output:
Starting syncing SourceAirtable
Marking stream talentclass/all_leads/tblnpRi7u0YECVout as STARTED
Syncing stream: talentclass/all_leads/tblnpRi7u0YECVout
Marking stream talentclass/all_leads/tblnpRi7u0YECVout as RUNNING
Exception ignored in:
Traceback (most recent call last):
File "/home/enrique/pyAirbyteTests/myenv/lib/python3.10/site-packages/airbyte/_batch_handles.py", line 84, in __del__
self.close_files()
File "/home/enrique/pyAirbyteTests/myenv/lib/python3.10/site-packages/airbyte/_batch_handles.py", line 66, in close_files
if self.open_file_writer is None:
File "/home/enrique/pyAirbyteTests/myenv/lib/python3.10/site-packages/airbyte/_batch_handles.py", line 62, in open_file_writer
return self._open_file_writer
AttributeError: 'BatchHandle' object has no attribute '_open_file_writer'

Read Progress

Started reading at 18:14:00.

Read 0 records over 3 seconds (0.0 records / second).
```

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.