apache / apache/buildstream

Credentials for different URLs isn't possible if they are under the same domain

Open
#2,148 5 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
140
Forks
45
Avg merge
1d 3h
Merged PRs (30d)
6

Description

## Problem

We have a buildstream project that uses sources from private repositories.

These repositories require different credentials, but they are all under the same domain.
This is a problem, as netrc matches domain name but not url paths.

There needs to be a different way to authenticate based on URLs that isn't using netrc.

## Potential solution

URLs could be matched based on globs. Services such as Gitlab lets you create tokens for a group, where any repositories under the group can use that token to pull the source or LFS tarballs.

These entries could be stored in user configuration, and I suggest prioritising authentication with it before using netrc as a fallback.
This wouldn't create compatibility issues, and allows for per-url authentication while still keeping a netrc file.

Example:

```yaml
# URL will be matched against each entry. If URL matches an entry, those credentials are used.
authentication:
# Match repository
- url: "https://example.com/group1/repo1"
username: "user1"
password: "pass1"
# Match repository group
- url: "https://example.com/group1/*"
username: "user2"
password: "pass2"
# Example of matching Gitlab LFS URL
# Note that escape sequence is used to escape the ? character
- url: 'https://example.com/api/v4/projects/group1%2Fnestedgroup%2Frepo/repository/files/lfs_object.tar.gz/raw\?ref=*&lfs=true'
username: "user3"
password: "user3"
```

### Additional notes for plugins

This solution would allow for plugins to also authenticate using user configuration, without having to implement a custom solution for authenticating per-url.

The following merge request was created by me for buildstream-plugins-community, where I implement a solution similar to what I've described above, but credentials are obtained from `~/.gitconfig`:

https://gitlab.com/BuildStream/buildstream-plugins-community/-/merge_requests/445.

Contributor guide

Open the contributing guide

Research direction

Start by tracing the existing netrc authentication path and user-configuration handling, then review the linked buildstream-plugins-community merge request for related behavior. Done means credentials can be selected by URL globs, user configuration takes precedence over netrc, fallback remains compatible, and plugins can use the same mechanism.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
authentication, build-system
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.