google / google/ml-metadata

GetArtifacts returns all artifacts without pagination

Open
#201 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
685
Forks
185
PR merge metrics
No merged PRs in 30d

Description

I'm using MLMD with MySQL DB + the official grpc server

When I don't give the option.max_result_size in a request, getArtifacts returns all artifacts without pagination

To reproduce it with `grpcurl`:
```
grpcurl -plaintext -proto ml_metadata/proto/metadata_store_service.proto localhost:13316 ml_metadata.MetadataStoreService/GetArtifacts
```

This is critical. When we have lots of artifacts (we're using Kubeflow), it will try to read all rows and it can kill a database instance.
Is this expected behavior? Shouldn't it return 100 artifacts with nextPageToken?

Another issue is when I give max_result_size larger than 100, nextPageToken will vanish and it always returns 101 artifacts.
I know I should not give a max_result_size larger than 100, but shouldn't it throw some errors?

https://github.com/google/ml-metadata/blob/master/ml_metadata/metadata_store/rdbms_metadata_access_object.cc#L2539

The if condition is only checking the lower bound.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.