Fail while pushing with --separate-weights
Open
Nobody has claimed this yet.
go
next
type/bug
- Dominant language
- Go
- Stars
- 9.5k
- Forks
- 696
- Avg merge
- 7d 19h
- Merged PRs (30d)
- 2
Description
Today I started to face the following issue, while using cog push --separate-weights
Although I was able to push the model without the flag --separate-weights
On the error below, I just replaced the project and model name by and <model-name, respectively.
$ cog push --separate-weights
⚠ Cog doesn't know if CUDA 11.2.2 is compatible with PyTorch 1.13.1. This might cause CUDA problems.
Building Docker image from environment in cog.yaml as r8.im/<project-name>/<model-name> ...
Weights unchanged, skip rebuilding and use cached image...
[+] Building 4.0s (7/7) FINISHED docker:default
=> [internal] load .dockerignore 0.0s
=> => transferring context: 22.25kB 0.0s
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 4.41kB 0.0s
=> resolve image config for docker.io/docker/dockerfile:1.4 1.6s
=> CACHED docker-image://docker.io/docker/dockerfile:1.4@sha256:9ba7531bd80fb0a858632727cf7a1 0.0s
=> [internal] load metadata for docker.io/nvidia/cuda:11.2.2-cudnn8-devel-ubuntu20.04 1.2s
=> ERROR [internal] load metadata for r8.im/<project-name>/<model-name> 2.2s
=> [auth] <project-name>/<model-name> -weights:pull token for r8.im 0.0s
------
> [internal] load metadata for r8.im/<project-name>/<model-name>-weights:latest:
------
Dockerfile:2
--------------------
1 | #syntax=docker/dockerfile:1.4
2 | >>> FROM r8.im/<project-name>/<model-name>-weights AS weights
3 | FROM nvidia/cuda:11.2.2-cudnn8-devel-ubuntu20.04
4 | ENV DEBIAN_FRONTEND=noninteractive
--------------------
ERROR: failed to solve: failed to authorize: failed to fetch oauth token: unexpected status from GET request to https://r8.im/_token?scope=repository%3A<project-name>%2F<model-name>-weights%3Apull&service=us-docker.pkg.dev: 404 Not Found
ⅹ Failed to build runner Docker image: Failed to build Docker image: exit status 1
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the cog push --separate-weights path and reproduce the Docker build failure shown in the issue. Inspect Dockerfile line 2, where the -weights image is used, and the registry token request that returns 404. Done means pushing with --separate-weights completes without the failed metadata lookup or authorization error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker
- Domain
- cli, devops
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100