airbytehq / airbytehq/PyAirbyte

Error when migrating data from mysql to duckdb

Aperta
#567 2 commenti 0 reazioni 0 assegnatari Vedi su GitHub
Lingua principale
Python
Stelle
344
Fork
77
Merge medio
1g 11h
PR unite (30g)
35

Descrizione

Hi! I'm trying to make PoC with pyairbyte and got some strange errors

Environment: Ubuntu 22, Python 3.11,

Errors
1. When I run script, it complains about docker but continues to run
```bash
AirbyteSubprocessFailedError: Subprocess failed.
Run Args: ['docker', 'run', '--rm', '-i', '--volume', '/home/aprytkov/test-airbyte/source-mysql:/local/', '--volume', '/tmp:/tmp', 'airbyte/source-mysql:latest', 'discover', '--config', '/tmp/tmp8hzu648g.json']
```

2. Full log: https://gist.github.com/Arkronus/5c5c424d3c458c2a20f4011397453683

Here is my code
```python
import airbyte as ab

def main():
# connectors_list = ab.get_available_connectors()
# print(f"Available connectors: {connectors_list}")

source_name = 'source-mysql'

mysql_source: ab.Source = ab.get_source(source_name, install_if_missing=True)

# open mysql databse https://docs.rfam.org/en/latest/database.html
mysql_source.set_config(
config={
"host":"mysql-rfam-public.ebi.ac.uk",
'port': 4497,
'database': 'Rfam',
'username': 'rfamro',
'replication_method': {
'method': 'STANDARD'
}
}
)

streams = mysql_source.get_available_streams()
print('Streams')
print(streams)

destination_name = 'destination-duckdb'

duckdb_destination: ab.Destination = ab.get_destination(destination_name, install_if_missing=True)
print(duckdb_destination.docs_url)
duckdb_destination.set_config(
config={
'destination_path': 'db.duckdb'
}
)

result = mysql_source.read(streams=['family','clan'])
duckdb_destination.write(result)
print("Load complete")

if __name__ == "__main__":
main()

```

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.