Add A Python-Notebook-Migration Tool
- Dominant language
- Scala
- Stars
- 314
- Forks
- 187
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 214
Description
### Document that tracks all related issues/PRs
https://docs.google.com/document/d/1BU4YxAgmWPmBE90UdIcZ38T8RAQ0q7voxPjrNwo8-3I/edit?usp=sharing
### Feature Summary
Currently, users who have existing code outside of Texera and want to migrate that code to Texera must create a workflow from scratch. This can take a long time to do depending on the complexity of the code. This tool aims to reduce the amount of time needed migrating to Texera by utilizing large language models to migrate Jupyter Notebooks to Texera workflows.
### Proposed Solution or Design
The following architecture diagrams and demo videos demonstrate the proposed tool.
#### Architecture Diagram
#### Demo Videos
1. User uploads a Jupyter Notebook using the new tool button. They can select any available model (depending on the LiteLLM configuration) and provide their API key. In the demo video, the API key is stored as a system environment variable, so it is left blank. After the user submits, they will need to wait until the LLM responds.
https://github.com/user-attachments/assets/d698dc40-939f-49a9-9439-de6ba3f11ca5
2. After the LLM responds, the workspace will reload with the generated workflow and the Jupyter popup.
https://github.com/user-attachments/assets/b471b334-2b42-46df-ae37-f834816f635f
3. The user can edit the workflow however they want. To assist with this process, when the user clicks on a workflow operator, it will highlight the corresponding notebook cell used to generate that operator, and vice versa.
https://github.com/user-attachments/assets/49ab77a2-0766-49ab-9faf-51b3488cfd03
4. The user can minimize the Jupyter panel and reopen it, and can also delete the notebook associated with the workflow.
https://github.com/user-attachments/assets/bc7c1b62-298d-4001-8de6-9bd28a43dec4
5. When the user reopens a workflow that was generated from a notebook, it will also reopen the notebook.
https://github.com/user-attachments/assets/ac94b8ba-cfb1-4c4e-9397-16779e2ec46a
### Related Discussions
Discussion #4590
### Impact / Priority
(P2)Medium – useful enhancement
### Affected Area
Other
Contributor guide
Assessment
This issue has not been assessed yet.