Can I just search by the metadata?
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 5k
- Forks
- 622
- Avg merge
- 8h 37m
- Merged PRs (30d)
- 5
Description
So here's my code. This returns something but I'm not sure why it returns it that way. Can somebody explain? This is based on the sample code from the github site.
`
using Microsoft.ANN.SPTAGManaged;
using System;
using System.IO;
using System.Text;
namespace SPTAG_Tester
{
class Program
{
static int dimension = 2;
static int n = 20;
static int k = 3;
static byte[] createFloatArray(int n)
{
byte[] data;
if (n == 1)
{
data = new byte[n * dimension * sizeof(float)];
var ee = 5;
for (int i = (0 + ee); i < (n + ee); i++)
{
for (int j = 0; j < dimension; j++)
{
var bb = BitConverter.GetBytes((float)i);
Array.Copy(bb, 0, data, ((i-ee) * dimension + j) * sizeof(float), 4);
}
}
var l = System.Text.Encoding.ASCII.GetString(data, 0, data.Length);
return data;
}
data = new byte[n * dimension * sizeof(float)];
for (int i = 0; i < n; i++)
for (int j = 0; j < dimension; j++)
Array.Copy(BitConverter.GetBytes((float)i), 0, data, (i * dimension + j) * sizeof(float), 4);
return data;
}
static byte[] createMetadata(int n)
{
StringBuilder sb = new StringBuilder();
sb.Append("zero\t65.0|220.0|\n");
sb.Append("one\t73.0|160.0|\n");
sb.Append("two\t59.0|110.0|\n");
sb.Append("three\t61.0|120.0|\n");
sb.Append("four\t75.0|150.0|\n");
sb.Append("five\t67.0|240.0|\n");
sb.Append("six\t68.0|230.0|\n");
sb.Append("seven\t70.0|220.0|\n");
sb.Append("eight\t62.0|130.0|\n");
sb.Append("nine\t66.0|210.0|\n");
sb.Append("ten\t77.0|190.0|\n");
sb.Append("11\t75.0|180.0|\n");
sb.Append("12\t74.0|170.0|\n");
sb.Append("13\t70.0|210.0|\n");
sb.Append("14\t61.0|110.0|\n");
sb.Append("15\t58.0|100.0|\n");
sb.Append("16\t66.0|230.0|\n");
sb.Append("17\t59.0|120.0|\n");
sb.Append("18\t68.0|210.0|\n");
sb.Append("19\t61.0|130.0|\n");
return Encoding.ASCII.GetBytes(sb.ToString());
}
static void Main()
{
{
AnnIndex idx = new AnnIndex("BKT", "Float", dimension);
idx.SetBuildParam("DistCalcMethod", "L2");
byte[] data = createFloatArray(n);
byte[] meta = createMetadata(n);
idx.BuildWithMetaData(data, meta, n, false);
idx.Save("testcsharp");
}
AnnIndex index = AnnIndex.Load("testcsharp");
var toSearch = createFloatArray(1);
BasicResult[] res = index.SearchWithMetaData(toSearch, k);
for (int i = 0; i < res.Length; i++)
Console.WriteLine("result num:" + i.ToString() +
": [" + res[i].Dist.ToString() + "(Distance)] " +
"@( [" + res[i].VID.ToString() + "(VectorID)] , " +
"[" + Encoding.ASCII.GetString(res[i].Meta).Replace("\n", string.Empty).Replace("\t", " ") + "(Metadata)])");
Console.WriteLine("test finish!");
Console.ReadLine();
}
}
}
`
The result is:
`
result num:0: [0(Distance)] @( [5(VectorID)] , [five 67.0|240.0|(Metadata)])
result num:1: [2(Distance)] @( [4(VectorID)] , [four 75.0|150.0|(Metadata)])
result num:2: [2(Distance)] @( [6(VectorID)] , [six 68.0|230.0|(Metadata)])
`
Why do I need to generate the 5th one in the data to get the VectorID 5? Isn't there a method to just search by the metadata id "five" instead of getting the byte[]'s by offsetting the index by 5?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the managed API entry points AnnIndex.BuildWithMetaData and SearchWithMetaData, then trace how metadata and VectorID are represented in the SPTAG implementation. Determine whether metadata identifiers can be queried directly rather than requiring a vector position, and document the supported behavior or the scope of an API change.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, csharp
- Domain
- api, search
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100