Add CD to publish PyPi Package
- Dominant language
- Python
- Stars
- 4.8k
- Forks
- 487
- PR merge metrics
- No merged PRs in 30d
Description
### 🚀 The feature, motivation, and pitch
It would be nice if we could `pip install trlx` in a project, rather than having to clone.
I suggest we use the approach that I also implemented for [TrasformerLens](https://github.com/neelnanda-io/TransformerLens):
- Use Poetry rather than SetupTools/PIP to manage dependencies when developing the project (end users can still use pip to install it). This automatically detects dependency clashes that can be a nightmare with PyPi releases. It also has some nice helper tooling (e.g. for bumping package versions). Downside is it's a bit slower.
- Break out checks from the pre-commit hook into a workflow that runs on all PRs and pushes to main (so that we can see which part failed and why) - things like pytest and mypi. Also run this on all common Python versions (a bit like https://github.com/neelnanda-io/TransformerLens/actions/runs/3934543895/jobs/6729387264 )
- Have a release workflow, so that when you create a release (e.g. with v.1.x.0) on GitHub it automatically bundles up the package and releases to PyPi (if the checks are still passing)
### Alternatives
Do all the above but use SetupTools still rather than Poetry. Advantage is it's still more known (although Poetry is very popular now), disadvantage is a lot of stuff is harder (e.g. bumping package versions in the CD automatically) and it doesn't check for conflicts.
### Additional context
Happy to implement this
Contributor guide
Research direction
Start by inspecting the existing pre-commit hook and package metadata, then review how pytest and mypy are currently run. Done means checks run across common Python versions on pull requests and pushes to main, and a release workflow builds and publishes an installable package to PyPI when a GitHub release is created.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions, python
- Domain
- build-system, devops, release, tooling
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100