apache / apache/wayang

add functionality required for this test load_collection & collect

Open
#631 0 comments 0 reactions 0 assignees View on GitHub
todo
Dominant language
Java
Stars
274
Forks
140
Avg merge
5d 16h
Merged PRs (30d)
4

Description

add functionality required for this test load_collection & collect

https://github.com/apache/incubator-wayang/blob/1e0f9e8166225176fe3022de5fbcce3dbcba96b9/python/src/pywy/tests/test_train_linear_svc.py#L24

```python

# limitations under the License.
#

import pytest

from pywy.dataquanta import WayangContext
from pywy.platforms.java import JavaPlugin
from pywy.platforms.spark import SparkPlugin

# TODO: add functionality required for this test load_collection & collect
@pytest.mark.skip(reason="no way of currently testing this, since we are missing implementations for load_collection & collect")
def test_train_and_predict():
ctx = WayangContext().register({JavaPlugin, SparkPlugin})

features = ctx.load_collection([[0.0, 1.0], [1.0, 0.0], [1.0, 1.0], [0.0, 0.0]])
labels = ctx.load_collection([1.0, 1.0, 0.0, 0.0])

model = features.train_linear_svc(labels, max_iter=10, reg_param=0.1)
predictions = model.predict(features)

result = predictions.collect()

print("Predictions:", result)

assert len(result) is 4, f"Expected len(result) to be 4, but got: {len(result)}"
for pred in result:
assert pred in [0.0, 1.0], f"Expected prediction to be in [0.0, 0.1], but got: {pred}"

```

6bd117a221c0fa18b566075684463e7058390aca

Contributor guide

Open the contributing guide

Research direction

Start with python/src/pywy/tests/test_train_linear_svc.py at the skipped test and trace the missing load_collection and collect entry points. Read the surrounding pywy dataquanta and registered JavaPlugin and SparkPlugin APIs before deciding where these operations belong. Done means the test can be unskipped and its predictions collection satisfies the stated assertions.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, python, spark
Domain
data-engineering, machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.