claimed-framework / claimed-framework/mlx_deprecated

Notebook registration with dependent files

Open
#162 0 comments 0 reactions 0 assignees View on GitHub
enhancement RCOS size/L
Dominant language
Python
Stars
220
Forks
54
PR merge metrics
No merged PRs in 30d

Description

MLX currently supports self contained notebooks.

MLX should allow users to upload a set of files along with the notebook metadata YAML like what Elyra is using for notebook definition: a set of Python files, that notebook imports. Or, settings files, requirements.txt, etc

We could store a set of files along with the YAML that is uploaded when a new notebook is registered in Minio. Those files we could then make available to Elyra for notebook execution.

We would either have to ask users to provide ...
- either, a S3 bucket with credentials that has all the files,
- or, a Github URL pointing to a repo that has them,
- or, a .tgz file that contains all the files

Ideally we extend the YAML metadata to list all of the dependent files, but that could also be a higher burden to put on users.

Reported/requested by @blublinsky

Contributor guide

Open the contributing guide

Research direction

Start by tracing how new notebooks are registered from their metadata YAML and stored in Minio, then inspect how Elyra receives files for execution. The issue leaves the source format and YAML design undecided, so first establish which dependency-upload approach the project will support. Done means registered notebooks can provide their dependent files to Elyra during execution.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.