Long-term - migrate observability -> datafusion

For small to medium sized projects the current span postgres backend; however, for large-scale, multi-tenant workflows with terabytes of data, we should consider revamping the datafusion logic

Plan: Create multi-worker datafusion/deltalake DB engine for observability data

Todos:

- [x] Create simple read/write logic with observability schema
- [ ] Support for attribute shreddling style logic for quick reads based on attributes (maybe map struct schema)
- [ ] Setup local buffer with periodic writes
- [x] Explore partitioning options for datafusion and/or deltalake (z order, etc)
- [x] Setup compaction jobs with deltalake
- [x] Setup vacuum jobs with deltalake
- [x] Build multi-worker system for distributed read/write architecture in order to support high levels of concurrency