iodide-project / iodide-project/iodide

proposal: `iodide.file.cache`? (name tbd)

Open
#1,838 6 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
1.5k
Forks
145
PR merge metrics
No merged PRs in 30d

Description

(context: discussion of "query" language plugin)

for workflows in which you need to run expensive computations or queries that you don't want to re-run when sharing a report, it is important that we have mechanisms for (a) not evaluating some code chunks, and (b) for saving and loading results of these expensive computations. currently, we have the ability to explicitly mark a chunk with `skipRunAll` to accomplish (a), and the `iodide.file.save` and `iodide.file.load` APIs to accomplish (b). In practice, these are used as follows:

```
%% {js, py, query, whatever} skipRunAll
// expensive computation, data munging, query that you
// don't want to re-run every time you load a report...
%% js skipRunAll
// runs when evaluated manually (during initial exploration)
// but does not run during run-all (e.g. when loading report view)
iodide.file.save("result.json", "json", result)
%% js
// does run when loading report view
iodide.file.load("result.json", "json", "result")
```

We could remove some of this verbosity (at the cost of being a little bit less explicit) by introducing a new function `iodide.file.cache` (or "cached" or "loadOrSave" or ????) with the same signature

`iodide.file.cache(fileIdentifierString, fileTypeStringFlag, varNameString)`
such that:
- if `varNameString` is not a valid js identifier, it throws an error.
- if `window[varNameString]` is undefined, it would load the data from the file at `fileIdentifierString` into `window[varNameString]`.
- if `window[varNameString]` is defined, it would save the data in `window[varNameString]` to the file at `fileIdentifierString`

then the above code example would shorten to:

```
%% {js, py, query, whatever} skipRunAll
// expensive computation, data munging, query that you
// don't want to re-run every time you load a report...
%% js
iodide.file.cache("result.json", "json", "result")
```

Any holes in this idea? @hamilton obviously you're closest to the implementation we have so far-- would this be feasible? any corner cases missing?

@rafrombrc @wlach @mdboom @teonbrooks -- general UX and naming ideas welcome.

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the existing iodide.file.save and iodide.file.load APIs and the skipRunAll behavior described in the issue. Determine whether a cache-style API is feasible, resolve naming and identifier-validation behavior, and document the corner cases and agreed semantics as the definition of done.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
api
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.