airbytehq / airbytehq/PyAirbyte

Airtable connector fails

Ouverte
#275 1 commentaire 3 réactions 1 personne assignée Réclamée par @aaronsteers Voir sur GitHub
Langage dominant
Python
Étoiles
344
Forks
77
Merge moyen
1 j 11 h
PR mergées (30 j)
35

Description

When reading from a successfully connected airtable connector, the connector looks for a file on the local cache, no file exists and the sync fails.

I tried to simplify the example as much as possible:

```
# Import PyAirbyte
import airbyte as ab

# Create and install airtable connector
source: ab.Source = ab.get_source("source-airtable")

# Configure the source
source.set_config(
config={
"credentials": {
"auth_method": "api_key",
"api_key": "MyKey"
},
},
)
# Verify the config and creds by running `check`:
source.check()

source.select_streams("*")

read_result: ab.ReadResult = source.read()

# Print the read result
print("Read result:")
print(read_result)
```

Error:

```
FileNotFoundError: [Errno 2] No such file or directory: '.cache/default_cache/talentclass/all_leads/tblnpRi7u0YECVout_01J01NPY9AKJ84BQKKK6S91BNR.jsonl.gz'
```

Full logs:

```
(myenv) enrique@Enrique-TUF:~/pyAirbyteTests$ python airtableTests.py

Connection check succeeded for `source-airtable`.
Started `source-airtable` read operation at 20:13:59...
Failed `source-airtable` read operation at 20:14:04.
Traceback (most recent call last):
File "/home/enrique/pyAirbyteTests/myenv/lib/python3.10/site-packages/airbyte/sources/base.py", line 734, in read
cache_processor.process_airbyte_messages(
File "/home/enrique/pyAirbyteTests/myenv/lib/python3.10/site-packages/airbyte/_future_cdk/record_processor.py", line 196, in process_airbyte_messages
self.process_record_message(
File "/home/enrique/pyAirbyteTests/myenv/lib/python3.10/site-packages/airbyte/_future_cdk/sql_processor.py", line 237, in process_record_message
self.file_writer.process_record_message(
File "/home/enrique/pyAirbyteTests/myenv/lib/python3.10/site-packages/airbyte/_processors/file/base.py", line 155, in process_record_message
batch_handle = self._new_batch(stream_name=stream_name)
File "/home/enrique/pyAirbyteTests/myenv/lib/python3.10/site-packages/airbyte/_processors/file/base.py", line 106, in _new_batch
batch_handle = BatchHandle(
File "/home/enrique/pyAirbyteTests/myenv/lib/python3.10/site-packages/airbyte/_batch_handles.py", line 30, in __init__
self._open_file_writer: IO[bytes] = file_opener(self._files[0])
File "/home/enrique/pyAirbyteTests/myenv/lib/python3.10/site-packages/airbyte/_processors/file/jsonl.py", line 34, in _open_new_file
return cast(IO[bytes], gzip.open(file_path, "w"))
File "/usr/lib/python3.10/gzip.py", line 58, in open
binary_file = GzipFile(filename, gz_mode, compresslevel)
File "/usr/lib/python3.10/gzip.py", line 174, in __init__
fileobj = self.myfileobj = builtins.open(filename, mode or 'rb')
FileNotFoundError: [Errno 2] No such file or directory: '.cache/default_cache/talentclass/all_leads/tblnpRi7u0YECVout_01J01NPY9AKJ84BQKKK6S91BNR.jsonl.gz'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/enrique/pyAirbyteTests/airtableTests.py", line 23, in
read_result: ab.ReadResult = source.read()
File "/home/enrique/pyAirbyteTests/myenv/lib/python3.10/site-packages/airbyte/sources/base.py", line 745, in read
raise exc.AirbyteConnectorFailedError(
airbyte.exceptions.AirbyteConnectorFailedError: AirbyteConnectorFailedError: Connector failed.

Log output:
Starting syncing SourceAirtable
Marking stream talentclass/all_leads/tblnpRi7u0YECVout as STARTED
Syncing stream: talentclass/all_leads/tblnpRi7u0YECVout
Marking stream talentclass/all_leads/tblnpRi7u0YECVout as RUNNING
Exception ignored in:
Traceback (most recent call last):
File "/home/enrique/pyAirbyteTests/myenv/lib/python3.10/site-packages/airbyte/_batch_handles.py", line 84, in __del__
self.close_files()
File "/home/enrique/pyAirbyteTests/myenv/lib/python3.10/site-packages/airbyte/_batch_handles.py", line 66, in close_files
if self.open_file_writer is None:
File "/home/enrique/pyAirbyteTests/myenv/lib/python3.10/site-packages/airbyte/_batch_handles.py", line 62, in open_file_writer
return self._open_file_writer
AttributeError: 'BatchHandle' object has no attribute '_open_file_writer'

Read Progress

Started reading at 18:14:00.

Read 0 records over 3 seconds (0.0 records / second).
```

Guide de contribution

Ouvrir le guide de contribution

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.