apache / apache/datafusion-sqlparser-rs

High compile time of crates using sqlparser(codegen phase) - any way to reduce generated code?​​

Open
#2,066 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
3.5k
Forks
772
Avg merge
4d 9h
Merged PRs (30d)
17

Description

In my project, some crates have very long compilation times (e.g., 40-50 seconds, with the codegen phase accounting for 70%-90% of that time). I suspect these crates are slow to compile due to their dependency on sqlparser.

I analyzed the compilation of one such crate using cargo llvm-lines. The top functions by lines are listed below:

```
Lines Copies Function name
----- ------ -------------
2180708 31532 (TOTAL)
220326 (10.1%, 10.1%) 21 (0.1%, 0.1%) ::visit
163970 (7.5%, 17.6%) 1995 (6.3%, 6.4%) as sqlparser::ast::visitor::VisitMut>::visit
117960 (5.4%, 23.0%) 3654 (11.6%, 18.0%) as sqlparser::ast::visitor::VisitMut>::visit
76507 (3.5%, 26.5%) 21 (0.1%, 18.0%) ::visit
```

As shown, the visitor trait implementations for the AST types generate a significant amount of code. Is there any way to reduce the code generation footprint of sqlparser?

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the reported compilation profile with cargo llvm-lines and inspect the generated VisitMut implementations for the AST types named in the report. Compare the codegen footprint and compilation time before and after any approach; done means a demonstrated reduction in generated code or compile time without breaking visitor behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
performance
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.