dbt-labs / dbt-labs/dbt

[Feature] <Speed Up dbt: Reduce Model Pick Time Efficiently>

Open
#11,421 3 comments 0 reactions 0 assignees View on GitHub
engine:v1 status:triage type:feature
Dominant language
Rust
Stars
13.8k
Forks
2.6k
Avg merge
21h 31m
Merged PRs (30d)
56

Description

### Is this your first time submitting a feature request?

- [x] I have read the [expectations for open source contributors](https://docs.getdbt.com/docs/contributing/oss-expectations)
- [x] I have searched the existing issues, and I could not find an existing issue for this feature
- [x] I am requesting a straightforward extension of existing dbt functionality, rather than a Big Idea better suited to a discussion

### Describe the feature

I am experiencing a performance issue while running dbt on my local machine. When executing the following command:

````
dbt compile --profile xyz -t dev -m +some_model --vars '{"some_long_list_vars"}'

13:02:48 Running with dbt=1.9.2
13:02:55 Registered adapter: databricks=1.9.4
13:02:55 Unable to do partial parsing because saved manifest not found. Starting full parse.
...
---- foo ---bar --logs-----------------------
------Significant delay before dbt selects the model for execution-----
---- foo ---bar --logs---------------------------

....
.....

13:05:42 Found 1599 models, 694 snapshots, 5000 data tests, 1457 sources, 837 macros, 1 unit test
13:05:42
13:05:42 Concurrency: 5 threads (target='some_env')
````

````
dbt --version

Core:
- installed: 1.9.2

The latest version of dbt-core could not be determined!
Make sure that the following URL is accessible:
https://pypi.org/pypi/dbt-core/json

Plugins:
- databricks: 1.9.4 - Update available!
- spark: 1.8.0 - Update available!

````

dbt takes approximately 3 minutes before picking up the model for execution, during which it is stuck in the parsing phase. This delay significantly impacts performance and development workflows.

### Describe alternatives you've considered

_No response_

### Who will this benefit?

_No response_

### Are you interested in contributing this feature?

_No response_

### Anything else?

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.