apache / apache/texera

Export the relational and sampling operators as Python

Open
#8,505 0 comments 0 reactions 1 assignee Claimed by @kz930 View on GitHub
Dominant language
Scala
Stars
314
Forks
187
Avg merge
1d 21h
Merged PRs (30d)
214

Description

### Task Summary

Sixteen operators that rearrange rows, rather than read what is inside one, still have no standalone form: the three joins, the three set operations, Aggregate, the sort family, Split, the two samplers, and If, Dummy and Sleep. A workflow built out of them exports a script of commented placeholders.

Each implements `generateStandaloneCode`, returning the pandas that does what its executor does.

The samplers need more than that. A sampler decides per row whether to keep it, so which rows survive is fixed by the exact sequence `java.util.Random` produces. Seeding Python's own generator, or numpy's, with the same seed selects a different set, and the script would then report a different sample than the workflow it came from. The generator is transcribed instead, in a helper the three operators that draw from one share.

Step 11 of 27 in #8325.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.