apache / apache/datafusion

[Blog] Async Scalar User Defined Functions

Open
#16,525 1 comment 2 reactions 1 assignee Claimed by @Adez017 View on GitHub
enhancement
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Is your feature request related to a problem or challenge?

@goldmedal added support for Async user defined functions in

- https://github.com/apache/datafusion/pull/14837

As @comphead points out, I don't know of any other database engine that supports this, so it would be a great idea to tell the world about it in a blog post

### Describe the solution you'd like

The following would make a killer blog post:

1. Overview of async user defined functions and their usecases (calling LLMs, fetching http results, etc)
2. Example of creating an async UDF with datafusion (perhaps calling out to run http requests to fetch wikipedia pages and write results to a parquet file)
4. Description of how it is implemneted (example plan with AsyncFuncExec, etc)

### Describe alternatives you've considered

_No response_

### Additional context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.