blaze / blaze/odo

live update on hdfstores in blaze server

Open
#53 1 comment 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
1k
Forks
131
PR merge metrics
No merged PRs in 30d

Description

@mrocklin @jreback

I would like to augment the blaze server so that if someone updates an hdfstore, the blaze server can do compute on the updated file.

Here is my plan, I'd like your feedback on it
- subclass hdfstore such that if timestamps change, the pytables file handle will be reopened if it's stale
- add an entry to resource to return this subclass

(maybe we call it livehdfstore?)

The other option is to look for a kwarg in the resource call (live=True?) and dispatch based on that to either the original hdfstore, or the subclass?

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by tracing the existing HDFStore implementation and the resource dispatch entry point. Determine how an externally updated HDF store can be detected and reopened safely, then add coverage showing that the Blaze server computes against updated data without requiring a restart.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.