akash-network / akash-network/console

perf(api): high baseline memory usage across API endpoints (RSS >512MB, heap >256MB)

Open
#2,589 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

performance tech-debt
Dominant language
TypeScript
Stars
264
Forks
94
Avg merge
18h 49m
Merged PRs (30d)
297

Description

Problem

Clinic doctor profiling reveals elevated memory usage across the majority of API endpoints. Out of 28 endpoints profiled, 17 exceeded the 512MB RSS threshold and 14 exceeded the 256MB heap threshold under 10 concurrent connections for 10s.

This is a systemic issue — not specific to any one endpoint — suggesting the server's baseline memory footprint is high before request handling even begins.

Worst offenders (RSS max)

Endpoint RSS max Heap max Clinic
/v1/provider-earnings/akash1example 779MB 588MB none
/v1/deployment/akash1example/1 760MB 566MB none
/v1/providers 634MB 255MB delay+cpu
/v1/graph-data/compute 586MB 395MB loop-util
/v1/gpu-prices 579MB 340MB delay+cpu
/v1/templates-list 576MB 195MB loop-util
/v1/proposals 569MB 235MB cpu+handles
/v1/nodes/mainnet 551MB 355MB none
/v1/network-capacity 547MB 351MB none
/v1/market-data 540MB 349MB none

Investigation areas

  • Startup allocations: What is the RSS/heap immediately after server boot, before any requests?
  • Connection pools: Are DB connection pools (Sequelize + Drizzle) over-allocated?
  • Cached objects: In-memory caches (Memoize decorators) may hold large provider/GPU lists that are never freed
  • Module loading: The bundled rest-app.js is 338KB — check if eager module initialization allocates large objects

Suggested next steps

  1. Profile server at idle (no requests) to establish baseline memory
  2. Check @Memoize decorated methods for unbounded cache growth
  3. Review connection pool sizes for both Sequelize and Drizzle
  4. Consider lazy initialization for large data structures

Context

Part of the API event loop performance audit. Full results in apps/api/profile-results.json.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with apps/api/profile-results.json and profile the server at idle to establish its RSS and heap baseline. Then inspect @Memoize methods, Sequelize and Drizzle pool settings, and eager initialization associated with rest-app.js. Done means the baseline and the principal source of the excessive memory usage are established with profiling evidence.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
api, backend, performance
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.