flyteorg / flyteorg/flyte

[Core feature] Image building support for multi-arch/buildx

Open
#5,822 14 comments 0 reactions 1 assignee Claimed by @featherchen View on GitHub
backlogged enhancement flytekit stale
Dominant language
Go
Stars
7.5k
Forks
886
Avg merge
1d 14h
Merged PRs (30d)
120

Description

### Motivation: Why do you think this is important?
I've not been able to get image spec files like

```yaml
platform: linux/amd64,linux/arm64
builder: default
```
to work in flytekit. The issue is that the default builder doesn't do multi-arch. I'm not sure if the `envd` builder does but the lack of caching support in envd makes it much harder to use.

### Goal: What should the final outcome look like, ideally?
We should support multi-arch images in flytekit, and I think we should do so by supporting buildx builders generally.

Support can be added under the current imagespec builder plugin (or a new plugin if that makes more sense). If the current plugin, then it'd look something like

```python
img_spec = ImageSpec(name="gpu", builder="default", options=DockerBuilder(name="my-builder"))
```

syntax can potentially change, cc @eapolinario who wrote this version, which will look for a buildx builder called `my-builder`

Flytekit should also be able to create a default buildx builder if one doesn't exist I think. But this can be a separate ticket too.

### Are you sure this issue hasn't been raised already?

- [X] Yes

### Have you read the Code of Conduct?

- [X] Yes

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.