apache / apache/datafusion-sqlparser-rs

Support `proc-macro` TokenStream

Open
#39 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
3.5k
Forks
772
Avg merge
4d 9h
Merged PRs (30d)
17

Description

I'm currently writing a proc macro that uses SQL to generate code that works with the storage pattern in [`specs`](https://github.com/slide-rs/specs). As is, I have to call `display` on the `TokenStream`, which converts it to a raw string. This will mean I'm undoing the tokenization for this parser to re-tokenize, while it should be possible for this parser to use the `TokenStream` directly.

This is low priority for me, as this issue only causes excess compilation time, but after creating this proc macro, I can add it to my own to-do.

Contributor guide

No contributing guide indexed for this repository

Research direction

The issue mentions a proc-macro TokenStream and the parser's current display-to-raw-string path, but no repository files or tests. Start by locating that input conversion and determine how direct TokenStream support can preserve tokenization; done means parsing proc-macro TokenStream without the display and re-tokenization step.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, sql
Domain
compilers
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.