airbytehq / airbytehq/PyAirbyte

Error when migrating data from mysql to duckdb

Abierto
#567 2 comentarios 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
Python
Estrellas
344
Forks
77
Merge medio
1 d 11 h
PR fusionados (30 d)
35

Descripción

Hi! I'm trying to make PoC with pyairbyte and got some strange errors

Environment: Ubuntu 22, Python 3.11,

Errors
1. When I run script, it complains about docker but continues to run
```bash
AirbyteSubprocessFailedError: Subprocess failed.
Run Args: ['docker', 'run', '--rm', '-i', '--volume', '/home/aprytkov/test-airbyte/source-mysql:/local/', '--volume', '/tmp:/tmp', 'airbyte/source-mysql:latest', 'discover', '--config', '/tmp/tmp8hzu648g.json']
```

2. Full log: https://gist.github.com/Arkronus/5c5c424d3c458c2a20f4011397453683

Here is my code
```python
import airbyte as ab

def main():
# connectors_list = ab.get_available_connectors()
# print(f"Available connectors: {connectors_list}")

source_name = 'source-mysql'

mysql_source: ab.Source = ab.get_source(source_name, install_if_missing=True)

# open mysql databse https://docs.rfam.org/en/latest/database.html
mysql_source.set_config(
config={
"host":"mysql-rfam-public.ebi.ac.uk",
'port': 4497,
'database': 'Rfam',
'username': 'rfamro',
'replication_method': {
'method': 'STANDARD'
}
}
)

streams = mysql_source.get_available_streams()
print('Streams')
print(streams)

destination_name = 'destination-duckdb'

duckdb_destination: ab.Destination = ab.get_destination(destination_name, install_if_missing=True)
print(duckdb_destination.docs_url)
duckdb_destination.set_config(
config={
'destination_path': 'db.duckdb'
}
)

result = mysql_source.read(streams=['family','clan'])
duckdb_destination.write(result)
print("Load complete")

if __name__ == "__main__":
main()

```

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.