dotnet / dotnet/machinelearning
Q: Faster way to Filter DataView
- Dominant language
- C#
- Stars
- 9.4k
- Forks
- 2k
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 11
Description
I filter data from a dataview to get all items within a specific time period.
It seems slow compared to filtering with LINQ from objects in memory. Is there a faster way to do it?
```c#
var boolFilter = df["timestamp"].ElementwiseGreaterThanOrEqual(unixStartTime);
var hourlydata = df.Filter(boolFilter);
var boolFilter2 = hourlydata["timestamp"].ElementwiseLessThan(unixEndTime);
hourlydata = hourlydata.Filter(boolFilter2);
```
In this example, I am creating predictions for a certain time period at a time.
In another example, I may need to filter by exact match. Normally, I might create a dictionary to help, but is there a way to support some type "indices" for DataViews?
Contributor guide
Assessment
This issue has not been assessed yet.