Allow UDFs to declare whether they are deterministic
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 306
- Forks
- 75
- PR merge metrics
- No merged PRs in 30d
Description
Currently we consider all UDFs as non-determinstic. This prevents engine from being able to push down predicates which can be resolved at query compile time. Presto/Hive/Spark allows UDFs to declare if they are deterministic. We should also provide something similar in our API.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the UDF API and how the framework represents UDF determinism across its supported engines. Compare the behavior described for Presto, Hive, and Spark, then define what the API should expose and how predicate pushdown should recognize it. Done means the API and compile-time predicate behavior are specified and covered for deterministic and non-deterministic UDFs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend-api-design, data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100