project-codeflare / project-codeflare/codeflare-sdk

[Feature] Integration test case with Data Science Pipeline, CodeFlare and KubeRay

Open
#425 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
35
Forks
66
Avg merge
48m
Merged PRs (30d)
7

Description

Name of Feature or Improvement

Create an integration test case to validate DSP, CodeFlare and KubeRay implementation.

Describe the Solution You Would Like to See
Test environment assumptions:
  1. Data Science Pipeline v1.
  2. Ray cluster shall consist of no more than 2 worker pods, with 2 CPU cores and less than 6 GB available for each pod.
  3. An integration test execution time shall be less than 20 mins in total.
  4. S3 storage may be available, if needed.
  5. Free of proprietary intellectual property.
  6. Public data only.
Proposed test case: Clustering text documents using k-means on scikit-learn education page.

https://scikit-learn.org/stable/auto_examples/text/plot_document_clustering.html

Data Science Pipeline stages:
  1. Downloading test data (https://scikit-learn.org/stable/auto_examples/text/plot_document_clustering.html#loading-text-data)
  2. Launch Ray cluster with two worker pods.
  3. Ray driver launches two Ray actors, deployed to a pod each. The first actor runs TfidfVectorizer, followed by Kmeans clustering and evaluation. The second actor runs HashingVectorizer, followed by Kmeans clustering and evaluation.
  4. Ray driver collects evaluation results from the two actors. Then it reports the summaries.
  5. Ray cluster is stopped and shutdown.
  6. Pipeline run is completed.
Expected test assets:
  1. DSP pipeline yaml to deploy and kick off test runs.
  2. Test image with Ray and document clustering code.
  3. CodeFlare image to deploy the test image.
  4. Preconfigured credentials and configmaps in the test environment.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the scikit-learn document-clustering example and map the six listed pipeline stages to the DSP YAML, test image, and CodeFlare image described in the issue. Done means a public-data, non-proprietary run completes in under 20 minutes within the stated Ray resource limits and reports both actor evaluations before shutdown.

Written by the indexing model from the issue text.

Assessment

Tech stack
kubernetes, python, scikit-learn
Domain
distributed-systems, machine-learning, testing-qa
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.