calkit / calkit/calkit

I want to be able to run my pipeline with Buildkite

Open
#918 1 comment 0 reactions 0 assignees View on GitHub
operators
Dominant language
Python
Stars
60
Forks
18
Avg merge
1d 16h
Merged PRs (30d)
38

Description

The value prop of Calkit is when you want to keep your artifacts with your project rather than using some external system for them. You version, push, and pull them like your code. This is important for research projects, since you want to know how, when, and where the evidence was created--the provenance.

To do this, we can compile the Calkit pipeline to a Buildkite pipeline. We can have a stage to pull, check envs, then run, saving copies of the artifacts in Buildkite if desired and managing the flow of them between stages. Lastly, we can commit and push.

Similar to the Calkit GH Action, we want to keep the pipeline and artifacts inside the project and be able to run anywhere, rather than relying on a horizontal platform for this sort of thing, since that's not modular enough.

Contributor guide

Open the contributing guide

Research direction

Start by reading the existing Calkit GitHub Action, which the issue identifies as the closest comparison, and review how Calkit currently compiles and runs pipelines. Then investigate the Buildkite pipeline model and define the required stages for pulling, checking environments, running, handling artifacts, and committing or pushing. Done means a Calkit pipeline can be run through Buildkite while keeping the pipeline and artifacts in the project.

Written by the indexing model from the issue text.

Assessment

Tech stack
git, python
Domain
ci-cd, devops
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.