kubesphere / kubesphere/ks-devops

When starting or creating a pipeline, the kubesphere alarm prompt 404

Open
#1,038 2 comments 0 reactions 0 assignees View on GitHub
kind/bug
Dominant language
Go
Stars
164
Forks
140
PR merge metrics
No merged PRs in 30d

Description

### What is version of KubeSphere DevOps has the issue?

devops-apiserver:ks-v3.4.1 ks-jenkins:v3.4.0-2.319.3-1

### How did you install the Kubernetes? Or what is the Kubernetes distribution?

My kubesphere has been upgraded to 3.4.1 after installing 3.3.1

### What happened?

Today, a colleague gave feedback that the assembly line could not run. I checked the logs on devops and found that the certificate had expired. Then, I tried to update the certificate for k8s (kubeadm certificates renew all), but it did not take effect. Then, I tried to upgrade kubeshee, but it still couldn't solve the problem. I went to the developer community to check( https://ask.kubesphere.io/forum/d/23239-kubesphere-jing-xiang-gou-jian-qi-s2ifu-wu-zheng-shu-guo-qi-wen-ti )
Fixed the issue of certificate expiration, but when operating on the watermark, it prompted the following message:
![image](https://github.com/kubesphere/ks-devops/assets/45881802/1a63a0df-9fa7-494c-8507-c94975321a15)
Then I checked the logs of devops-apiserver and kept prompting the following content:
![image](https://github.com/kubesphere/ks-devops/assets/45881802/b831416b-92b8-4823-bd4d-aa81f768142b)
Then I checked the logs of devops Jenkins and found that the user wukai1 does not exist:

![image](https://github.com/kubesphere/ks-devops/assets/45881802/babb6b7b-2187-4855-99a1-53bc9826aeee)
But there is no user wukai1 in kubeshpere,

![image](https://github.com/kubesphere/ks-devops/assets/45881802/99e05522-0a58-40c7-9446-d72e7e3c4113)
![image](https://github.com/kubesphere/ks-devops/assets/45881802/57a29bb9-7758-48e2-b228-80222343d4d7)

### Relevant log output

```shell
2024-04-07 14:40:13.390+0000 [id=143] WARNING h.i.i.InstallUncaughtExceptionHandler#handleException: Caught unhandled exception with ID 6d1c45c0-a747-44b8-b37a-0074f07ab25e

org.acegisecurity.userdetails.UsernameNotFoundException: User wukai1 not found in directory.

at org.acegisecurity.ldap.search.FilterBasedLdapUserSearch.searchForUser(FilterBasedLdapUserSearch.java:126)

at hudson.security.LDAPSecurityRealm$LDAPUserDetailsService.loadUserByUsername(LDAPSecurityRealm.java:1320)

at hudson.security.LDAPSecurityRealm$LDAPUserDetailsService.loadUserByUsername(LDAPSecurityRealm.java:1273)

at org.acegisecurity.userdetails.UserDetailsService.lambda$toSpring$1(UserDetailsService.java:52)

Caused: org.springframework.security.core.userdetails.UsernameNotFoundException

at org.acegisecurity.userdetails.UsernameNotFoundException.toSpring(UsernameNotFoundException.java:51)

at org.acegisecurity.userdetails.UsernameNotFoundException.toSpring(UsernameNotFoundException.java:34)

at org.acegisecurity.userdetails.UserDetailsService.lambda$toSpring$1(UserDetailsService.java:54)

at jenkins.security.ImpersonatingUserDetailsService2.loadUserByUsername(ImpersonatingUserDetailsService2.java:29)

at hudson.model.User.getUserDetailsForImpersonation2(User.java:406)

at hudson.model.User.getUserDetailsForImpersonation(User.java:429)

Caused: org.acegisecurity.userdetails.UsernameNotFoundException: org.springframework.security.core.userdetails.UsernameNotFoundException: org.acegisecurity.userdetails.UsernameNotFoundException: User wukai1 not found in directory.; nested exception is org.springframework.security.core.userdetails.UsernameNotFoundException: org.acegisecurity.userdetails.UsernameNotFoundException: User wukai1 not found in directory.

at org.acegisecurity.userdetails.UsernameNotFoundException.fromSpring(UsernameNotFoundException.java:58)

at org.acegisecurity.AuthenticationException.fromSpring(AuthenticationException.java:98)

at hudson.model.User.getUserDetailsForImpersonation(User.java:431)

at io.kubesphere.jenkins.devops.auth.KubesphereApiTokenAuthenticator.authenticate(KubesphereApiTokenAuthenticator.java:47)

Caused: javax.servlet.ServletException

at io.kubesphere.jenkins.devops.auth.KubesphereApiTokenAuthenticator.authenticate(KubesphereApiTokenAuthenticator.java:54)

at jenkins.security.BasicHeaderAuthenticator.authenticate2(BasicHeaderAuthenticator.java:43)

at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:83)

at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:97)

at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:110)

at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:80)

at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:62)

at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:97)

at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:109)

at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:171)

at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)

at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)

at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:51)

at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)

at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)

at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:85)

at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)

at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)

at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)

at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)

at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)

at jenkins.security.SuspiciousRequestFilter.doFilter(SuspiciousRequestFilter.java:39)

at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)

at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)

at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548)

at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)

at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:578)

at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)

at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)

at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624)

at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)

at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1434)

at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)

at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501)

at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594)

at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)

at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1349)

at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)

at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)

at org.eclipse.jetty.server.Server.handle(Server.java:516)

at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:388)

at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:633)

at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:380)

at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)

at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)

at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)

at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)

at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)

at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)

at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)

at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)

at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:386)

at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)

at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)

at java.base/java.lang.Thread.run(Thread.java:829)
```

### Additional information

```markdown
Kubernetes:v1.22.12
kubesphere: 3.4.1(member:3.3.1)
devops-apiserver:ks-v3.4.1
ks-jenkins:v3.4.0-2.319.3-1
other:dex:v2.29.0(Connected to Gilab and logged into Kubeshpere through Gilab account)
```

Contributor guide

Open the contributing guide

Research direction

Start with the reported KubesphereApiTokenAuthenticator.authenticate stack-trace entry point and the devops-apiserver and Jenkins logs. Reproduce starting or creating a pipeline with the affected certificate and identity setup, then trace the 404 and missing wukai1 user behavior. Done means the pipeline operation no longer returns the alarm prompt or the reported authentication failure.

Written by the indexing model from the issue text.

Assessment

Tech stack
gitlab, kubernetes
Domain
authentication, ci-cd, devops
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.