argoproj / argoproj/argo-workflows

3.4 SSO RBAC with managed namespace works differently

Open
#9,989 27 comments 9 reactions 0 assignees View on GitHub
area/sso-rbac P1 solution/workaround type/regression
Dominant language
Go
Stars
17k
Forks
3.7k
Avg merge
1d 20h
Merged PRs (30d)
138

Description

### Pre-requisites

- [X] I have double-checked my configuration
- [X] I can confirm the issues exists when I tested with `:latest`
- [ ] I'd like to contribute the fix myself (see [contributing guide](https://github.com/argoproj/argo-workflows/blob/master/docs/CONTRIBUTING.md))

### What happened/what you expected to happen?

I upgraded Argo Workflow from 3.1.13 to 3.4.3. SSO Authentication was working fine with 3.1.13; however, the 3.4.3 doesn't seem to work.
The SSO configuration (Okta) has not changed.

When I tried to open the UI and click on login on the SSO, I get a red banner on the down right corner saying `Failed to load version/info Error: Unauthorized`. After that the web page just tries to load and after sometime it replies with `test-ce-argo-server-integration.k8s.cnqr.tech didn't send any data.ERR_EMPTY_RESPONSE`

### Version

3.4.3

### Paste a small workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflows that uses private images.

```YAML
N/A - The issue happens at login time, so I can't run any workflow.
```

### Logs from the workflow ~controller~ server

I am attaching the logs of the workflow server, because the error happens during authentication:

```
time="2022-11-07T13:37:32.115Z" level=info duration=2.752873ms method=GET path=/main.2430295409b8b54e52ad.js size=1471060 status=0
time="2022-11-07T13:37:32.117Z" level=info duration="23.156µs" method=GET path=index.html size=0 status=304
time="2022-11-07T13:37:33.935Z" level=info msg="finished unary call with code Unauthenticated" error="rpc error: code = Unauthenticated desc = token not valid for running mode" grpc.code=Unauthenticated grpc.method=GetUserInfo grpc.service=info.InfoService grpc.start_time="2022-11-07T13:37:33Z" grpc.time_ms=0.047 span.kind=server system=grpc
time="2022-11-07T13:37:33.935Z" level=info msg="finished unary call with code Unauthenticated" error="rpc error: code = Unauthenticated desc = token not valid for running mode" grpc.code=Unauthenticated grpc.method=GetInfo grpc.service=info.InfoService grpc.start_time="2022-11-07T13:37:33Z" grpc.time_ms=0.028 span.kind=server system=grpc
time="2022-11-07T13:37:33.935Z" level=info duration=1.628208ms method=GET path=/api/v1/userinfo size=56 status=401
time="2022-11-07T13:37:33.935Z" level=info duration=2.284157ms method=GET path=/api/v1/info size=56 status=401
time="2022-11-07T13:37:34.116Z" level=info duration="202.575µs" method=GET path=/assets/fonts/fa-solid-900.woff2 size=150472 status=0
time="2022-11-07T13:37:34.116Z" level=info duration="111.014µs" method=GET path=/assets/images/logo.png size=41464 status=0
time="2022-11-07T13:37:34.389Z" level=info msg="finished unary call with code Unauthenticated" error="rpc error: code = Unauthenticated desc = token not valid for running mode" grpc.code=Unauthenticated grpc.method=CollectEvent grpc.service=info.InfoService grpc.start_time="2022-11-07T13:37:34Z" grpc.time_ms=0.03 span.kind=server system=grpc
time="2022-11-07T13:37:34.389Z" level=info duration="438.543µs" method=POST path=/api/v1/tracking/event size=56 status=401
time="2022-11-07T13:37:36.819Z" level=info duration="67.668µs" method=GET path=index.html size=473 status=0
time="2022-11-07T13:37:56.819Z" level=info duration="68.792µs" method=GET path=index.html size=473 status=0
time="2022-11-07T13:38:16.819Z" level=info duration="74.393µs" method=GET path=index.html size=473 status=0
time="2022-11-07T13:38:36.819Z" level=info duration="83.064µs" method=GET path=index.html size=473 status=0
time="2022-11-07T13:38:56.819Z" level=info duration="68.599µs" method=GET path=index.html size=473 status=0
time="2022-11-07T13:39:16.819Z" level=info duration="81.42µs" method=GET path=index.html size=473 status=0
time="2022-11-07T13:39:36.819Z" level=info duration="72.698µs" method=GET path=index.html size=473 status=0
```

### Logs from in your workflow's wait container

N/A

This is the service account configured for RBAC:

```yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: admin-user
annotations:
# The rule is an expression used to determine if this service account
# should be used.
# * `groups` - an array of the OIDC groups
# * `iss` - the issuer ("argo-server")
# * `sub` - the subject (typically the username)
# Must evaluate to a boolean.
# If you want an account to be the default to use, this rule can be "true".
# Details of the expression language are available in
# https://github.com/antonmedv/expr/blob/master/docs/Language-Definition.md.
workflows.argoproj.io/rbac-rule: "true"
# The precedence is used to determine which service account to use whe
# Precedence is an integer. It may be negative. If omitted, it defaults to "0".
# Numerically higher values have higher precedence (not lower, which maybe
# counter-intuitive to you).
# If two rules match and have the same precedence, then which one used will
# be arbitrary.
workflows.argoproj.io/rbac-rule-precedence: "0"
```

Contributor guide

Open the contributing guide

Research direction

Start with the workflow server authentication path for the failing GetUserInfo and GetInfo calls, using the /api/v1/userinfo and /api/v1/info logs and the supplied RBAC service-account configuration. Reproduce the 3.4.3 SSO login failure and compare it with 3.1.13; done means the UI login succeeds and these endpoints no longer return 401 for the managed namespace.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kubernetes
Domain
authentication, authorization, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.