elastic / elastic/ml-cpp

[ML] Report actual memory usage for pytorch_inference process

Open
#2,885 1 comment 0 reactions 1 assignee Claimed by @darius-vil View on GitHub
:ml >enhancement
Dominant language
C++
Stars
157
Forks
67
Avg merge
12h 48m
Merged PRs (30d)
16

Description

Add the capability to report the actual OS memory usage (RSS) for the `pytorch_inference` process, similar to what was implemented for `autodetect` in #2846.

**Background**

PR #2846 introduced reporting of actual memory usage (via `getrusage` RSS) for the `autodetect` process. This provides valuable insight into the real memory footprint of anomaly detection jobs as reported by the OS, rather than relying solely on internal memory tracking.

The `pytorch_inference` process currently:
- Has the infrastructure to report RSS values via `writeProcessStats()` (called on-demand via `E_ProcessStats` control message)
- Uses `CProcessStats::residentSetSize()` and `CProcessStats::maxResidentSetSize()` in `Main.cc`
- Does **not** periodically report this information back to the Java process

**Proposed Changes**

1. Add periodic reporting of system memory usage for `pytorch_inference`, similar to how `autodetect` updates `E_TSADSystemMemoryUsage` and `E_TSADMaxSystemMemoryUsage` program counters.

2. Include the RSS values in the output stream that can be consumed by the Java side. Options include:
- Adding new fields to an existing result type
- Creating a new periodic stats message
- Extending the response from `E_ProcessStats` to be sent periodically

3. The values to report:
- `system_memory_bytes` - current resident set size (`CProcessStats::residentSetSize()`)
- `max_system_memory_bytes` - peak resident set size (`CProcessStats::maxResidentSetSize()`)

**Files likely to be modified**

- `bin/pytorch_inference/Main.cc` - Add periodic memory reporting
- `bin/pytorch_inference/CResultWriter.cc` / `CResultWriter.h` - Potentially extend output format
- `bin/pytorch_inference/CCommandParser.cc` / `CCommandParser.h` - If new message types are needed

**Relates to**

- elastic/ml-cpp#2846 (autodetect actual memory reporting)
- elastic/elasticsearch#139233 (Java-side changes for trained model stats)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.