CarperAI / CarperAI/trlx

Add CD to publish PyPi Package

Open
#199 4 comments 0 reactions 0 assignees View on GitHub
feature request
Dominant language
Python
Stars
4.8k
Forks
487
PR merge metrics
No merged PRs in 30d

Description

### 🚀 The feature, motivation, and pitch

It would be nice if we could `pip install trlx` in a project, rather than having to clone.

I suggest we use the approach that I also implemented for [TrasformerLens](https://github.com/neelnanda-io/TransformerLens):

- Use Poetry rather than SetupTools/PIP to manage dependencies when developing the project (end users can still use pip to install it). This automatically detects dependency clashes that can be a nightmare with PyPi releases. It also has some nice helper tooling (e.g. for bumping package versions). Downside is it's a bit slower.
- Break out checks from the pre-commit hook into a workflow that runs on all PRs and pushes to main (so that we can see which part failed and why) - things like pytest and mypi. Also run this on all common Python versions (a bit like https://github.com/neelnanda-io/TransformerLens/actions/runs/3934543895/jobs/6729387264 )
- Have a release workflow, so that when you create a release (e.g. with v.1.x.0) on GitHub it automatically bundles up the package and releases to PyPi (if the checks are still passing)

### Alternatives

Do all the above but use SetupTools still rather than Poetry. Advantage is it's still more known (although Poetry is very popular now), disadvantage is a lot of stuff is harder (e.g. bumping package versions in the CD automatically) and it doesn't check for conflicts.

### Additional context

Happy to implement this

Contributor guide

Open the contributing guide

Research direction

Start by inspecting the existing pre-commit hook and package metadata, then review how pytest and mypy are currently run. Done means checks run across common Python versions on pull requests and pushes to main, and a release workflow builds and publishes an installable package to PyPI when a GitHub release is created.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, python
Domain
build-system, devops, release, tooling
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.