dask / dask/dask-ml

Inclusion of databases?

Open
#896 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
951
Forks
262
PR merge metrics
No merged PRs in 30d

Description

Howdy Folks,

I see that parallel processing stuff and linear regression is supported to work distributedly
from an input which is a csv file.
I want to execute a clustering algorithm with data which is stored in database. Yeah I know there is a huge downside to actually
just downloading the data of a file, since the query operations might take time (they are simple) and the transmission from server to client even more.
But is there a possibility to actually use as input from rows the databases and whilst caching data (Vectors for examples) in the working memory or/and store the important metrics of the algorithm in database table?

Contributor guide

Open the contributing guide

Research direction

The report names no file, test, or entry point. Start by reviewing existing Dask-ML data-loading and clustering integrations, then clarify whether database-backed input, caching, and metric persistence are all in scope. Completion criteria are not defined in the issue.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
databases, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.