kubeflow / kubeflow/sdk

KEP-107: Convert Spark SDK Python examples to Jupyter notebooks

Open
#635 6 comments 0 reactions 1 assignee Claimed by @Neha-Nimsatkar View on GitHub
area/spark
Dominant language
Python
Stars
148
Forks
262
Avg merge
1d 2h
Merged PRs (30d)
1

Description

## Summary

Convert the Spark SDK examples from standalone Python scripts into Jupyter notebooks, similar to the Trainer examples.

## Motivation

Many Spark users work primarily in Jupyter notebooks, especially within Kubeflow Notebooks. Notebook-based examples provide a more interactive and reproducible experience, allowing users to execute each step incrementally while following the workflow.

This would also align the Spark SDK examples with the Trainer examples and provide a foundation for future Kubeflow Notebooks v2 documentation.

## Proposed work

- Convert existing Spark SDK examples into Jupyter notebooks.
- Preserve the existing example workflows while organizing them into executable notebook cells.
- Include explanatory markdown cells describing each step.
- Ensure the notebooks are reproducible and easy to run in a Kubeflow environment.

## Benefits

- Improved onboarding experience for new users.
- Interactive, step-by-step examples.
- Better alignment with Trainer examples.
- Useful reference material for future Kubeflow Notebooks documentation.

## Context

This idea came from the discussion https://github.com/kubeflow/sdk/pull/629#discussion_r3634829461, where notebook-based examples similar to the Trainer examples were suggested as a better user experience.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.