apache / apache/livy

Implementing Redaction logic to logs

Open
#491 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Scala
Stars
963
Forks
625
Avg merge
6d 21h
Merged PRs (30d)
6

Description

I noticed that request object is being logged here which contains the body of the request sent to livy

here - https://github.com/apache/incubator-livy/blob/f3e23ed0b34da9d44ec06c724d5d03c7cf15aebe/server/src/main/scala/org/apache/livy/server/interactive/InteractiveSession.scala#L119

I would like to implement a redaction logic using spark.redaction.regex that is used for a spark session to stop sensitive information from being exposed in logs

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at server/src/main/scala/org/apache/livy/server/interactive/InteractiveSession.scala around line 119, where the request object is logged. Read how Spark uses spark.redaction.regex, then define and verify that sensitive request-body information is redacted from Livy logs.

Written by the indexing model from the issue text.

Assessment

Tech stack
scala, spark
Domain
security
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.