[Feature] Apply selection only once when running `dbt build`
- Dominant language
- Rust
- Stars
- 13.8k
- Forks
- 2.6k
- Avg merge
- 21h 31m
- Merged PRs (30d)
- 56
Description
### Is this your first time submitting a feature request?
- [X] I have read the [expectations for open source contributors](https://docs.getdbt.com/docs/contributing/oss-expectations)
- [X] I have searched the existing issues, and I could not find an existing issue for this feature
- [X] I am requesting a straightforward extension of existing dbt functionality, rather than a Big Idea better suited to a discussion
### Describe the feature
When running `dbt` commands with `--select` criteria against a project with thousands of models, their resolution can be pretty slow - e.g. one minute for one intersection. But when running `dbt build` the situation is even worse, because the resolution happens three times (once per each underlying command).
I wonder if `dbt build` could trigger `dbt ls` under-the-hood as an initial step, collect the output, and pass the list of found resources to the following commands instead of re-applying `--select`s each time. Potentially it should result in a significant performance improvement.
Sample logs:
```
[2024-08-20, 06:23:28 UTC] {subprocess.py:93} INFO - 06:23:28 Found 14340 models, 31287 data tests, 3 seeds, 2 operations, 7686 sources, 1549 macros
[2024-08-20, 06:36:35 UTC] {subprocess.py:93} INFO - 06:36:35 The selection criterion '+tag:tag-a,tag:tag-b,config.materialized:view,result:error' does not match any enabled nodes
[2024-08-20, 06:37:33 UTC] {subprocess.py:93} INFO - 06:37:33 The selection criterion '+tag:tag-a,tag:tag-b,config.materialized:view,result:skipped' does not match any enabled nodes
[2024-08-20, 06:38:31 UTC] {subprocess.py:93} INFO - 06:38:31 The selection criterion '+tag:tag-a,tag:tag-b,config.materialized:view,result:fail' does not match any enabled nodes
[2024-08-20, 06:51:37 UTC] {subprocess.py:93} INFO - 06:51:37 The selection criterion '+tag:tag-a,tag:tag-b,config.materialized:view,result:error' does not match any enabled nodes
[2024-08-20, 06:52:36 UTC] {subprocess.py:93} INFO - 06:52:36 The selection criterion '+tag:tag-a,tag:tag-b,config.materialized:view,result:skipped' does not match any enabled nodes
[2024-08-20, 06:53:35 UTC] {subprocess.py:93} INFO - 06:53:35 The selection criterion '+tag:tag-a,tag:tag-b,config.materialized:view,result:fail' does not match any enabled nodes
[2024-08-20, 07:06:45 UTC] {subprocess.py:93} INFO - 07:06:45 The selection criterion '+tag:tag-a,tag:tag-b,config.materialized:view,result:error' does not match any enabled nodes
[2024-08-20, 07:07:44 UTC] {subprocess.py:93} INFO - 07:07:44 The selection criterion '+tag:tag-a,tag:tag-b,config.materialized:view,result:skipped' does not match any enabled nodes
[2024-08-20, 07:08:43 UTC] {subprocess.py:93} INFO - 07:08:43 The selection criterion '+tag:tag-a,tag:tag-b,config.materialized:view,result:fail' does not match any enabled nodes
```
Contributor guide
Assessment
This issue has not been assessed yet.