airbytehq / airbytehq/PyAirbyte

Error when migrating data from mysql to duckdb

未关闭
#567 2 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
主要语言
Python
星标
344
派生
77
平均合并
1 天 11 小时
30 天内合并 PR
35

描述

Hi! I'm trying to make PoC with pyairbyte and got some strange errors

Environment: Ubuntu 22, Python 3.11,

Errors
1. When I run script, it complains about docker but continues to run
```bash
AirbyteSubprocessFailedError: Subprocess failed.
Run Args: ['docker', 'run', '--rm', '-i', '--volume', '/home/aprytkov/test-airbyte/source-mysql:/local/', '--volume', '/tmp:/tmp', 'airbyte/source-mysql:latest', 'discover', '--config', '/tmp/tmp8hzu648g.json']
```

2. Full log: https://gist.github.com/Arkronus/5c5c424d3c458c2a20f4011397453683

Here is my code
```python
import airbyte as ab

def main():
# connectors_list = ab.get_available_connectors()
# print(f"Available connectors: {connectors_list}")

source_name = 'source-mysql'

mysql_source: ab.Source = ab.get_source(source_name, install_if_missing=True)

# open mysql databse https://docs.rfam.org/en/latest/database.html
mysql_source.set_config(
config={
"host":"mysql-rfam-public.ebi.ac.uk",
'port': 4497,
'database': 'Rfam',
'username': 'rfamro',
'replication_method': {
'method': 'STANDARD'
}
}
)

streams = mysql_source.get_available_streams()
print('Streams')
print(streams)

destination_name = 'destination-duckdb'

duckdb_destination: ab.Destination = ab.get_destination(destination_name, install_if_missing=True)
print(duckdb_destination.docs_url)
duckdb_destination.set_config(
config={
'destination_path': 'db.duckdb'
}
)

result = mysql_source.read(streams=['family','clan'])
duckdb_destination.write(result)
print("Load complete")

if __name__ == "__main__":
main()

```

贡献指南

打开贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。