apache / apache/seatunnel

[Feature][Zeta] Bound the size of REST log-content responses

Open
#12,295 1 comment 0 reactions 1 assignee Claimed by @SEZ9 View on GitHub
Dominant language
Java
Stars
9.7k
Forks
2.4k
Avg merge
3d 17h
Merged PRs (30d)
210

Description

### Search before asking

- [X] I had searched in the [feature](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22Feature%22) and found no similar feature requirement.

### Description

The REST log-content endpoints read the whole file into a `String` before writing the response, with no upper bound:

```java
// LogBaseServlet#prepareLogResponse
String logContent = FileUtils.readFileToStr(new File(canonicalFilePath).toPath());
write(resp, logContent);
```

```java
// FileUtils#readFileToStr
byte[] bytes = Files.readAllBytes(path);
return new String(bytes);
```

`RestHttpGetCommandProcessor` carries an equivalent copy for REST v1.

That means a single `GET /logs/` on a long-running streaming job's log file materialises the entire file as a byte array **and** again as a `String` on the heap. A multi-gigabyte log — normal for a job that has been streaming for weeks — can push the node into a long GC pause or an `OutOfMemoryError`, taking down the whole engine node rather than just failing the request.

The existing path-traversal guard in `prepareLogResponse` is intact; this is purely about response size.

### Usage Scenario

An operator opens a job log through the Web UI or `curl`s the log endpoint on a node whose log files have grown large. Today that is enough to destabilise the node. There is no configuration knob to prevent it.

Proposal: add `seatunnel.engine.http.log-response-max-size-mb` (default large enough to be a safety net rather than a behaviour change) and return the **tail** of the file when it exceeds the limit — the tail being the useful part when diagnosing a failure. The tail must be aligned to a line boundary, otherwise slicing at an arbitrary byte offset splits a multi-byte UTF-8 character and the response starts with a replacement character.

### Related issues

_No response_

### Are you willing to submit a PR?

- [X] Yes I am willing to submit a PR!

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.