apache / apache/wayang

add functionality required for this test load_collection & collect

Open
#633 1 comment 0 reactions 0 assignees View on GitHub
todo
Dominant language
Java
Stars
274
Forks
140
Avg merge
5d 16h
Merged PRs (30d)
4

Description

add functionality required for this test load_collection & collect

https://github.com/apache/incubator-wayang/blob/1e0f9e8166225176fe3022de5fbcce3dbcba96b9/python/src/pywy/tests/train_logistic_test.py#L24

```python

# limitations under the License.
#

import pytest

from pywy.dataquanta import WayangContext
from pywy.platforms.java import JavaPlugin
from pywy.platforms.spark import SparkPlugin

# TODO: add functionality required for this test load_collection & collect
@pytest.mark.skip(reason="no way of currently testing this, since we are missing implementations for load_collection & collect")
def test_train_and_predict():
ctx = WayangContext().register({JavaPlugin, SparkPlugin})

features = ctx.load_collection([[0.0, 1.0], [1.0, 0.0], [1.0, 1.0], [0.0, 0.0]])
labels = ctx.load_collection([1.0, 1.0, 0.0, 0.0])

model = features.train_logistic_regression(labels)
predictions = model.predict(features)

result = predictions.collect()
print("Predictions:", result)

assert len(result) is 4, f"Expected len(result) to be 4, but got: {len(result)}"
for pred in result:
assert pred in [0.0, 1.0], f"Expected prediction to be in [0.0, 0.1], but got: {pred}"

```

f35294bf9a27cc63281f965c1bf55a42419e4b35

Contributor guide

Open the contributing guide

Research direction

Start with python/src/pywy/tests/train_logistic_test.py at the skipped test_train_and_predict test and review the referenced load_collection and collect calls. Trace the existing WayangContext, JavaPlugin, and SparkPlugin implementations to determine the missing functionality. Done means the test can run without its skip marker and its collection and prediction assertions pass.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, python, spark
Domain
data-engineering, testing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.