developmentseed / developmentseed/bioacoustics-api
Try using DiskANN
- Dominant language
- Jupyter Notebook
- Stars
- 1
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
Milvus supports on-disk index https://milvus.io/docs/disk_index.md. This should reduce compute usage drastically. DiskANN is enabled by default but I think we need to make sure the volume inside the container is mounted at `/var/lib/milvus/data`
I think we'll want to add a [non-boot persistent disk](https://cloud.google.com/compute/docs/disks/persistent-disks) to the cluster and then have k8s mount that to the query nodes and index nodes. We'll need to use `DISKANN` as the index type while building the index.
@leothomas Once you have the index update done #54, let's figure out a plan here. Might be good to create a second indexing script/job that can create the DiskANN index that has a volume mounted.
I should be able to quickly adjust the k8s configuration to support this and get some help @sunu too.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.