sameersbn / sameersbn/docker-gitlab
Cannot use S3 with GITLAB_ARTIFACTS_OBJECT_STORE
Nobody has claimed this yet.
- Dominant language
- Shell
- Stars
- 8.1k
- Forks
- 2.1k
- Avg merge
- 17h 47m
- Merged PRs (30d)
- 7
Description
There seem to be some issue with GITLAB_ARTIFACTS_OBJECT_STORE_* parameters. It seems to not be working correctly with S3 URLs.
We're using the following parameters
- GITLAB_ARTIFACTS_OBJECT_STORE_ENABLED=true
- GITLAB_ARTIFACTS_OBJECT_STORE_REMOTE_DIRECTORY="sample-bucket-name"
- GITLAB_ARTIFACTS_OBJECT_STORE_DIRECT_UPLOAD=false
- GITLAB_ARTIFACTS_OBJECT_STORE_PROXY_DOWNLOAD=true
- GITLAB_ARTIFACTS_OBJECT_STORE_BACKGROUND_UPLOAD=true
- GITLAB_ARTIFACTS_OBJECT_STORE_CONNECTION_AWS_PATH_STYLE=false
- GITLAB_ARTIFACTS_OBJECT_STORE_CONNECTION_PROVIDER=AWS
- GITLAB_ARTIFACTS_OBJECT_STORE_CONNECTION_AWS_ACCESS_KEY_ID=******
- GITLAB_ARTIFACTS_OBJECT_STORE_CONNECTION_AWS_SECRET_ACCESS_KEY=******
- GITLAB_ARTIFACTS_OBJECT_STORE_CONNECTION_AWS_REGION=us-east-1
- GITLAB_ARTIFACTS_OBJECT_STORE_CONNECTION_AWS_HOST="s3.us-east-1.amazonaws.com"
In this case when trying to run rake gitlab:artifacts:migrate we'll get
bad URI(is not URI?): "://sample-bucket-name.s3.amazonaws.com:"
This URL doesn't seem correct since it's missing a region, it's even more interesting because we specified GITLAB_ARTIFACTS_OBJECT_STORE_CONNECTION_AWS_HOST and in theory it should use it when forming URL, but it does not. We tried removing double quotes around GITLAB_ARTIFACTS_OBJECT_STORE_CONNECTION_AWS_HOST, we tried not using GITLAB_ARTIFACTS_OBJECT_STORE_CONNECTION_AWS_HOST at all, we tried using GITLAB_ARTIFACTS_OBJECT_STORE_CONNECTION_AWS_PATH_STYLE=true instead of false, none of these seem to give correct URI for the S3.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing rake gitlab:artifacts:migrate with the listed GITLAB_ARTIFACTS_OBJECT_STORE settings. Trace how the artifact object-store configuration constructs the S3 URL, especially the host, region, and path-style options. Done means migration forms a valid S3 URI and completes successfully.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, docker
- Domain
- cloud, devops
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100