[Feature] <Speed Up dbt: Reduce Model Pick Time Efficiently>
- Dominant language
- Rust
- Stars
- 13.8k
- Forks
- 2.6k
- Avg merge
- 21h 31m
- Merged PRs (30d)
- 56
Description
### Is this your first time submitting a feature request?
- [x] I have read the [expectations for open source contributors](https://docs.getdbt.com/docs/contributing/oss-expectations)
- [x] I have searched the existing issues, and I could not find an existing issue for this feature
- [x] I am requesting a straightforward extension of existing dbt functionality, rather than a Big Idea better suited to a discussion
### Describe the feature
I am experiencing a performance issue while running dbt on my local machine. When executing the following command:
````
dbt compile --profile xyz -t dev -m +some_model --vars '{"some_long_list_vars"}'
13:02:48 Running with dbt=1.9.2
13:02:55 Registered adapter: databricks=1.9.4
13:02:55 Unable to do partial parsing because saved manifest not found. Starting full parse.
...
---- foo ---bar --logs-----------------------
------Significant delay before dbt selects the model for execution-----
---- foo ---bar --logs---------------------------
....
.....
13:05:42 Found 1599 models, 694 snapshots, 5000 data tests, 1457 sources, 837 macros, 1 unit test
13:05:42
13:05:42 Concurrency: 5 threads (target='some_env')
````
````
dbt --version
Core:
- installed: 1.9.2
The latest version of dbt-core could not be determined!
Make sure that the following URL is accessible:
https://pypi.org/pypi/dbt-core/json
Plugins:
- databricks: 1.9.4 - Update available!
- spark: 1.8.0 - Update available!
````
dbt takes approximately 3 minutes before picking up the model for execution, during which it is stuck in the parsing phase. This delay significantly impacts performance and development workflows.
### Describe alternatives you've considered
_No response_
### Who will this benefit?
_No response_
### Are you interested in contributing this feature?
_No response_
### Anything else?
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.