[Bug] Pulsar Authorization with Keycloak
- Dominant language
- Java
- Stars
- 15.3k
- Forks
- 3.8k
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 160
Description
### Version
OS: Ubuntu 22.04
Pulsar version: v3.0.1
docker image: "apachepulsar/pulsar:latest"
### Minimal reproduce step
My docker -compose file:
```
version: "3.5"
services:
pulsar:
image: "apachepulsar/pulsar:latest"
container_name: pulsar
command: bin/pulsar standalone
environment:
PULSAR_MEM: " -Xms512m -Xmx512m -XX:MaxDirectMemorySize=1g"
ports:
- "6650:6650"
- "8081:8080"
restart: unless-stopped
```
After I run the pulsar container i exec and configure my `pulsar/conf/standalone.conf file`
```
authenticationEnabled=true
authenticationProviders=org.apache.pulsar.broker.authentication.AuthenticationProviderToken
authorizationEnabled=false
authorizationProvider=org.apache.pulsar.broker.authorization.PulsarAuthorizationProvider
brokerClientAuthenticationPlugin=org.apache.pulsar.client.impl.auth.oauth2.AuthenticationOAuth2
brokerClientAuthenticationParameters={"privateKey":"file:///pulsar/oauth2.json","audience":"MyAuthApp","issuerUrl":"{keycloak-server-url}/realms/{myrealm}"}
tokenPublicKey=file:///pulsar/oauth_public.key
```
My oauth2.json file:
```
{
"type": "client_credentials",
"client_id": "MyAuthApp",
"client_secret": "{client-secret}",
"issuer_url": "https://{keycloak-server-url}/realms/{myrealm}"
}
```
### What did you expect to see?
Authentication works perfectly, but when I enable authorization and restart my pulsar container I get exceptions:
`
authorizationEnabled=true`
### What did you see instead?
```
pulsar | 2023-11-29T09:51:07,574+0000 [main] ERROR org.apache.pulsar.broker.PulsarService - Failed to start Pulsar service: org.apache.pulsar.client.admin.PulsarAdminException$NotAuthorizedException: Unauthorized to validateNamespaceOperation for operation [CREATE_TOPIC] on namespace [public/functions]
pulsar | java.lang.RuntimeException: org.apache.pulsar.client.admin.PulsarAdminException$NotAuthorizedException: Unauthorized to validateNamespaceOperation for operation [CREATE_TOPIC] on namespace [public/functions]
pulsar | at org.apache.pulsar.functions.worker.PulsarWorkerService.start(PulsarWorkerService.java:584) ~[org.apache.pulsar-pulsar-functions-worker-3.0.1.jar:3.0.1]
pulsar | at org.apache.pulsar.broker.PulsarService.startWorkerService(PulsarService.java:1797) ~[org.apache.pulsar-pulsar-broker-3.0.1.jar:3.0.1]
pulsar | at org.apache.pulsar.broker.PulsarService.start(PulsarService.java:890) ~[org.apache.pulsar-pulsar-broker-3.0.1.jar:3.0.1]
pulsar | at org.apache.pulsar.PulsarStandalone.start(PulsarStandalone.java:349) ~[org.apache.pulsar-pulsar-broker-3.0.1.jar:3.0.1]
pulsar | at org.apache.pulsar.PulsarStandaloneStarter.main(PulsarStandaloneStarter.java:141) ~[org.apache.pulsar-pulsar-broker-3.0.1.jar:3.0.1]
pulsar | Caused by: org.apache.pulsar.client.admin.PulsarAdminException$NotAuthorizedException: Unauthorized to validateNamespaceOperation for operation [CREATE_TOPIC] on namespace [public/functions]
pulsar | at org.apache.pulsar.client.admin.PulsarAdminException.wrap(PulsarAdminException.java:252) ~[org.apache.pulsar-pulsar-client-admin-api-3.0.1.jar:3.0.1]
pulsar | at org.apache.pulsar.client.admin.internal.BaseResource.sync(BaseResource.java:352) ~[org.apache.pulsar-pulsar-client-admin-original-3.0.1.jar:3.0.1]
pulsar | at org.apache.pulsar.client.admin.internal.TopicsImpl.createNonPartitionedTopic(TopicsImpl.java:308) ~[org.apache.pulsar-pulsar-client-admin-original-3.0.1.jar:3.0.1]
pulsar | at org.apache.pulsar.client.admin.Topics.createNonPartitionedTopic(Topics.java:539) ~[org.apache.pulsar-pulsar-client-admin-api-3.0.1.jar:3.0.1]
pulsar | at org.apache.pulsar.functions.worker.PulsarWorkerService.tryCreateNonPartitionedTopic(PulsarWorkerService.java:387) ~[org.apache.pulsar-pulsar-functions-worker-3.0.1.jar:3.0.1]
pulsar | at org.apache.pulsar.functions.worker.PulsarWorkerService.start(PulsarWorkerService.java:440) ~[org.apache.pulsar-pulsar-functions-worker-3.0.1.jar:3.0.1]
pulsar | ... 4 more
pulsar | Suppressed: org.apache.pulsar.client.admin.PulsarAdminException$NotAuthorizedException: Unauthorized to validateNamespaceOperation for operation [CREATE_TOPIC] on namespace [public/functions]
pulsar | at org.apache.pulsar.client.admin.internal.BaseResource.getApiException(BaseResource.java:281) ~[org.apache.pulsar-pulsar-client-admin-original-3.0.1.jar:3.0.1]
pulsar | at org.apache.pulsar.client.admin.internal.BaseResource$1.failed(BaseResource.java:136) ~[org.apache.pulsar-pulsar-client-admin-original-3.0.1.jar:3.0.1]
pulsar | at org.glassfish.jersey.client.JerseyInvocation$1.failed(JerseyInvocation.java:882) ~[org.glassfish.jersey.core-jersey-client-2.34.jar:?]
pulsar | at org.glassfish.jersey.client.JerseyInvocation$1.completed(JerseyInvocation.java:863) ~[org.glassfish.jersey.core-jersey-client-2.34.jar:?]
pulsar | at org.glassfish.jersey.client.ClientRuntime.processResponse(ClientRuntime.java:229) ~[org.glassfish.jersey.core-jersey-client-2.34.jar:?]
pulsar | at org.glassfish.jersey.client.ClientRuntime.access$200(ClientRuntime.java:62) ~[org.glassfish.jersey.core-jersey-client-2.34.jar:?]
pulsar | at org.glassfish.jersey.client.ClientRuntime$2.lambda$response$0(ClientRuntime.java:173) ~[org.glassfish.jersey.core-jersey-client-2.34.jar:?]
pulsar | at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) ~[org.glassfish.jersey.core-jersey-common-2.34.jar:?]
pulsar | at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) ~[org.glassfish.jersey.core-jersey-common-2.34.jar:?]
pulsar | at org.glassfish.jersey.internal.Errors.process(Errors.java:292) ~[org.glassfish.jersey.core-jersey-common-2.34.jar:?]
pulsar | at org.glassfish.jersey.internal.Errors.process(Errors.java:274) ~[org.glassfish.jersey.core-jersey-common-2.34.jar:?]
pulsar | at org.glassfish.jersey.internal.Errors.process(Errors.java:244) ~[org.glassfish.jersey.core-jersey-common-2.34.jar:?]
pulsar | at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:288) ~[org.glassfish.jersey.core-jersey-common-2.34.jar:?]
pulsar | at org.glassfish.jersey.client.ClientRuntime$2.response(ClientRuntime.java:173) ~[org.glassfish.jersey.core-jersey-client-2.34.jar:?]
pulsar | at org.apache.pulsar.client.admin.internal.http.AsyncHttpConnector.lambda$apply$1(AsyncHttpConnector.java:254) ~[org.apache.pulsar-pulsar-client-admin-original-3.0.1.jar:3.0.1]
pulsar | at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) ~[?:?]
pulsar | at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) ~[?:?]
pulsar | at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510) ~[?:?]
pulsar | at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2147) ~[?:?]
pulsar | at org.apache.pulsar.client.admin.internal.http.AsyncHttpConnector.lambda$retryOperation$4(AsyncHttpConnector.java:296) ~[org.apache.pulsar-pulsar-client-admin-original-3.0.1.jar:3.0.1]
pulsar | at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) ~[?:?]
pulsar | at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) ~[?:?]
pulsar | at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510) ~[?:?]
pulsar | at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2147) ~[?:?]
pulsar | at org.asynchttpclient.netty.NettyResponseFuture.loadContent(NettyResponseFuture.java:222) ~[org.asynchttpclient-async-http-client-2.12.1.jar:?]
pulsar | at org.asynchttpclient.netty.NettyResponseFuture.done(NettyResponseFuture.java:257) ~[org.asynchttpclient-async-http-client-2.12.1.jar:?]
pulsar | at org.asynchttpclient.netty.handler.AsyncHttpClientHandler.finishUpdate(AsyncHttpClientHandler.java:241) ~[org.asynchttpclient-async-http-client-2.12.1.jar:?]
pulsar | at org.asynchttpclient.netty.handler.HttpHandler.handleChunk(HttpHandler.java:114) ~[org.asynchttpclient-async-http-client-2.12.1.jar:?]
pulsar | at org.asynchttpclient.netty.handler.HttpHandler.handleRead(HttpHandler.java:143) ~[org.asynchttpclient-async-http-client-2.12.1.jar:?]
pulsar | at org.asynchttpclient.netty.handler.AsyncHttpClientHandler.channelRead(AsyncHttpClientHandler.java:78) ~[org.asynchttpclient-async-http-client-2.12.1.jar:?]
pulsar | at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[io.netty-netty-transport-4.1.93.Final.jar:4.1.93.Final]
pulsar | at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[io.netty-netty-transport-4.1.93.Final.jar:4.1.93.Final]
pulsar | at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[io.netty-netty-transport-4.1.93.Final.jar:4.1.93.Final]
pulsar | at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[io.netty-netty-codec-4.1.93.Final.jar:4.1.93.Final]
pulsar | at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[io.netty-netty-transport-4.1.93.Final.jar:4.1.93.Final]
pulsar | at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[io.netty-netty-transport-4.1.93.Final.jar:4.1.93.Final]
pulsar | at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[io.netty-netty-transport-4.1.93.Final.jar:4.1.93.Final]
pulsar | at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436) ~[io.netty-netty-transport-4.1.93.Final.jar:4.1.93.Final]
pulsar | at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) ~[io.netty-netty-codec-4.1.93.Final.jar:4.1.93.Final]
pulsar | at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) ~[io.netty-netty-codec-4.1.93.Final.jar:4.1.93.Final]
pulsar | at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251) ~[io.netty-netty-transport-4.1.93.Final.jar:4.1.93.Final]
pulsar | at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) ~[io.netty-netty-transport-4.1.93.Final.jar:4.1.93.Final]
pulsar | at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[io.netty-netty-transport-4.1.93.Final.jar:4.1.93.Final]
pulsar | at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[io.netty-netty-transport-4.1.93.Final.jar:4.1.93.Final]
pulsar | at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[io.netty-netty-transport-4.1.93.Final.jar:4.1.93.Final]
pulsar | at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) ~[io.netty-netty-transport-4.1.93.Final.jar:4.1.93.Final]
pulsar | at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[io.netty-netty-transport-4.1.93.Final.jar:4.1.93.Final]
pulsar | at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[io.netty-netty-transport-4.1.93.Final.jar:4.1.93.Final]
pulsar | at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) ~[io.netty-netty-transport-4.1.93.Final.jar:4.1.93.Final]
pulsar | at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) ~[io.netty-netty-transport-4.1.93.Final.jar:4.1.93.Final]
pulsar | at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) ~[io.netty-netty-transport-4.1.93.Final.jar:4.1.93.Final]
pulsar | at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) ~[io.netty-netty-transport-4.1.93.Final.jar:4.1.93.Final]
pulsar | at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) ~[io.netty-netty-transport-4.1.93.Final.jar:4.1.93.Final]
pulsar | at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[io.netty-netty-common-4.1.93.Final.jar:4.1.93.Final]
pulsar | at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[io.netty-netty-common-4.1.93.Final.jar:4.1.93.Final]
pulsar | at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[io.netty-netty-common-4.1.93.Final.jar:4.1.93.Final]
pulsar | at java.lang.Thread.run(Thread.java:833) ~[?:?]
pulsar | Caused by: javax.ws.rs.ForbiddenException: HTTP 403 {"reason":"Unauthorized to validateNamespaceOperation for operation [CREATE_TOPIC] on namespace [public/functions]"}
pulsar | at org.glassfish.jersey.client.JerseyInvocation.convertToException(JerseyInvocation.java:945) ~[org.glassfish.jersey.core-jersey-client-2.34.jar:?]
pulsar | at org.glassfish.jersey.client.JerseyInvocation.access$700(JerseyInvocation.java:82) ~[org.glassfish.jersey.core-jersey-client-2.34.jar:?]
pulsar | ... 54 more
```
### Anything else?
After searching online to find a solution for this exception I tried changing this parameter, but it did not work:
`authorizationProvider=org.apache.pulsar.broker.authorization.MultiRolesTokenAuthorizationProvider`
Contributor guide
Research direction
Start with the standalone.conf settings and the docker-compose reproduction, then trace the authorization failure from PulsarWorkerService.start for CREATE_TOPIC on public/functions. Reproduce the startup failure with authorization enabled and compare the authentication and authorization configuration; done means standalone Pulsar starts successfully with the intended authorization behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, docker-compose, java
- Domain
- authentication, authorization, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100