apache / apache/kyuubi

[FEATURE] Allow to dump user queries within kyuubi server for audit

Open
#4,181 0 comments 1 reaction 0 assignees View on GitHub
kind:feature priority:major
Dominant language
Scala
Stars
2.4k
Forks
1k
PR merge metrics
No merged PRs in 30d

Description

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)

### Search before asking

- [X] I have searched in the [issues](https://github.com/apache/kyuubi/issues?q=is%3Aissue) and found no similar issues.

### Describe the feature

While ranger is able to save the users queries somewhere, the kyuubi server (AFAIK) has no way to do this by itself.

### Motivation

Having this audit feature within kyuubi would provide more security feature without using ranger.

### Describe the solution

add a configuration with a path (hdfs/s3) like
`kyuubi.audit.path=s3://my-audit-bucket/`

as a result, for each user queries, a sql dump would be stored on path:

`s3://my-audit-bucket/user1/date/time/query-id.sql`

### Additional context

_No response_

### Are you willing to submit PR?

- [X] Yes. I would be willing to submit a PR with guidance from the Kyuubi community to improve.
- [ ] No. I cannot submit a PR at this time.

Contributor guide

Open the contributing guide

Research direction

The issue names no files, tests, or entry points; start by tracing how the Kyuubi server receives user SQL and how configuration handles HDFS or S3 paths. Done means a configured audit path stores each user's query as a SQL dump under the described user/date/time/query-id layout.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, hadoop, scala
Domain
backend, distributed-systems, security
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.