apache / apache/sedona-db

dev/release: Simplify cargo publish

Open
#702 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
503
Forks
61
Avg merge
2d 4h
Merged PRs (30d)
90

Description

Our cargo publish sequence is currently quite complicated because we have circular dev dependencies. It's possible to work around this but it is annoying and not great practice!

I think we should:

- Add some check for circular dependencies in the dependency graph. The only culprits I know about are `sedona` and `sedona-testing` in dev-dependencies.
- I don't know how sedona-raster became a dependency of sedona-functions but we should probably move that
- sedona depending on sedona-testing (for the random geometry generator) needs to get fixed.
- Separate out a crate for high-level benchmarks and/or high level tests. DataFusion has most of its tests in the `datafusion` crate's `tests` directory and a completely separate benchmark setup, perhaps for this reason. I do like keeping *most* of the tests file-local and I'm sure we can find a way to separate out the logic we need (maybe a high- and low- level testing crate) to keep that (even if we have to move some higher level integration tests or benchmarks).
- Many of our benchmarks could be dynamic (e.g., we could have a CLI that passes some ArgSpecs as strings).

Contributor guide

Open the contributing guide

Research direction

Start by tracing the current cargo publish sequence and dependency graph, focusing on the sedona and sedona-testing dev-dependencies and the sedona-raster dependency of sedona-functions. Review the proposed crate separation for high-level tests and benchmarks, then verify that publishing no longer requires circular-dependency workarounds and that the affected tests and benchmarks still run.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
build-system, release
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.