[CT-530] [Bug] Partial parsing should handle volatile variables like `invocation_id` and `run_started_at`
- Dominant language
- Rust
- Stars
- 13.8k
- Forks
- 2.6k
- Avg merge
- 21h 31m
- Merged PRs (30d)
- 56
Description
### Is there an existing issue for this?
- [X] I have searched the existing issues
### Current Behavior
You can trick dbt into giving dynamic freshness check behaviour with something like this:
```yml
freshness:
error_after:
count: "{{ 48 if run_started_at.strftime('%w') == '1' else 24 }}" #looser expectations on Monday
period: hour
```
[Community Slack thread](https://getdbt.slack.com/archives/C2JRTGLLS/p1638216863082300).
@jtcohen6 pointed out that
>Small word of warning — this won't be re-parsed by partial parsing, and so it's liable to get stale. Shouldn't be an issue in scheduled runs (which don't persist partial-parse state between runs), it might just be cause for surprise if you come back to it locally a few days from now.
https://docs.getdbt.com/reference/parsing#known-limitations
### Expected Behavior
Again from jerco:
>In the future, we should think about ways to always re-parse (invalidate cache) for any configs/properties/nodes/etc that depend on "volatile" variables like run_started_at or invocation_id
### Steps To Reproduce
1. Configure a source freshness check as described above, such that it will fail today and pass every other day of the week
2. Run `dbt source freshness`, note that the test fails
3. Wait a day (or change the code above to be dependent on the minute of the day or something)
4. Run `dbt source freshness` again. Note that the test still fails, even though it's a different day of the week.
5. Run `dbt source freshness --no-partial-parse`. Note that the test now passes, because it's re-parsed the project and re-evaluated `run_started_at`.
### Relevant log output
_No response_
### Environment
```markdown
- OS: macOS
- Python: 3.8.x
- dbt: 1.0-rc1
```
### What database are you using dbt with?
snowflake
### Additional Context
Related: https://github.com/dbt-labs/dbt-core/issues/2330
Contributor guide
Assessment
This issue has not been assessed yet.