airbytehq / airbytehq/PyAirbyte

Error when migrating data from mysql to duckdb

Ouverte
#567 2 commentaires 0 réactions 0 personnes assignées Voir sur GitHub
Langage dominant
Python
Étoiles
344
Forks
77
Merge moyen
1 j 11 h
PR mergées (30 j)
35

Description

Hi! I'm trying to make PoC with pyairbyte and got some strange errors

Environment: Ubuntu 22, Python 3.11,

Errors
1. When I run script, it complains about docker but continues to run
```bash
AirbyteSubprocessFailedError: Subprocess failed.
Run Args: ['docker', 'run', '--rm', '-i', '--volume', '/home/aprytkov/test-airbyte/source-mysql:/local/', '--volume', '/tmp:/tmp', 'airbyte/source-mysql:latest', 'discover', '--config', '/tmp/tmp8hzu648g.json']
```

2. Full log: https://gist.github.com/Arkronus/5c5c424d3c458c2a20f4011397453683

Here is my code
```python
import airbyte as ab

def main():
# connectors_list = ab.get_available_connectors()
# print(f"Available connectors: {connectors_list}")

source_name = 'source-mysql'

mysql_source: ab.Source = ab.get_source(source_name, install_if_missing=True)

# open mysql databse https://docs.rfam.org/en/latest/database.html
mysql_source.set_config(
config={
"host":"mysql-rfam-public.ebi.ac.uk",
'port': 4497,
'database': 'Rfam',
'username': 'rfamro',
'replication_method': {
'method': 'STANDARD'
}
}
)

streams = mysql_source.get_available_streams()
print('Streams')
print(streams)

destination_name = 'destination-duckdb'

duckdb_destination: ab.Destination = ab.get_destination(destination_name, install_if_missing=True)
print(duckdb_destination.docs_url)
duckdb_destination.set_config(
config={
'destination_path': 'db.duckdb'
}
)

result = mysql_source.read(streams=['family','clan'])
duckdb_destination.write(result)
print("Load complete")

if __name__ == "__main__":
main()

```

Guide de contribution

Ouvrir le guide de contribution

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.