[BUG] Can not load vertices from parquet
- Dominant language
- C++
- Stars
- 3.6k
- Forks
- 468
- Avg merge
- 29m
- Merged PRs (30d)
- 1
Description
**Describe the bug**
When loading a graph from local Parquet files mounted via hostPath volume into a K8s session, GraphScope fails with Object not exists in vineyard followed by Empty table list cannot be used for normalizing schema. The same setup works correctly with CSV files.
**To Reproduce**
- Set up a K8s cluster (tested with K3s on single node Ubuntu)
- Mount a local directory containing Parquet files as a hostPath volume
- Create a GraphScope session with k8s_volumes pointing to the mount
- Call add_vertices with a file:/// path to a .parquet file
```
import graphscope as gs
with gs.session(k8s_volumes={
"data": {
"type": "hostPath",
"field": {
"path": "/path/to/host/dataset",
"type": "Directory"
},
"mounts": {
"mountPath": "/dataset"
}
}
}, k8s_image_tag='0.30.0', k8s_namespace='gs') as session:
graph = session.g()
graph = graph.add_vertices(
'file:///dataset/vertices.parquet',
label='person',
properties=['firstName', 'lastName', 'gender'],
vid_field='id'
)
graph = graph.add_edges(
'file:///dataset/edges.parquet',
src_label='person',
dst_label='person'
)
```
**Expected behavior**
Graph should load from Parquet files the same way it loads from CSV files.
**Environment**
- GraphScope version: v0.30.0
- vineyard version: v0.24.2
- OS: Ubuntu 24.04
- Kubernetes: K3s v1.24.9+k3s1(single node)
- Python 3.10.20
- Graphscope python library: v0.29.0
**Additional context**
I faced the the same issue if load dataset from S3:
```
import graphscope as gs
from graphscope.framework.loader import Loader
import argparse
parser = argparse.ArgumentParser(description="Process command-line arguments.")
parser.add_argument("--s3-key", type=str)
parser.add_argument("--s3-secret", type=str)
args = parser.parse_args()
# Doesn't work
with gs.session(k8s_namespace='gs', k8s_image_tag='0.31.0') as session:
print(session.info)
graph = session.g()
print("Graph loaded")
graph = graph.add_vertices(Loader('s3://bucket-name/csv/vertices.csv', delimiter='|', key=args.s3_key, secret=args.s3_secret, endpoint_url='https://custom-endpoint.ru'), label='person')
print("Vertices loaded")
graph = graph.add_edges(Loader('s3://bucket-name/csv/edges.csv', delimiter='|', key=args.s3_key, secret=args.s3_secret, endpoint_url='https://custom-endpoint.ru'), src_label='person', dst_label='person')
print("Edges loaded")
```
[k8s.logs.txt](https://github.com/user-attachments/files/26134890/k8s.logs.txt)
[python.logs.txt](https://github.com/user-attachments/files/26134889/python.logs.txt)
Contributor guide
Assessment
This issue has not been assessed yet.