OHDSI / OHDSI/WebAPI

Memory Usage WebAPI

Open
#2,358 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Java
Stars
151
Forks
183
Avg merge
14m
Merged PRs (30d)
2

Description

When WebAPI is started on our kubernetes cluster it is currently using 1168Mi of memory when idle whereas Hades is using 75Mi and Atlas is using 4Mi. This makes WebAPI relatively expensive

NAMESPACE             NAME                                               CPU(cores)   MEMORY(bytes)   
cert-manager          cert-manager-6c69f9f796-hx4xb                      1m           22Mi            
cert-manager          cert-manager-cainjector-584f44558c-29cwl           1m           33Mi            
cert-manager          cert-manager-webhook-78466c75fc-frwrv              1m           24Mi            
cert-manager          trust-manager-775bfcf747-nwzxq                     2m           28Mi            
flux-system           helm-controller-5d8d5fc6fd-6pn5r                   2m           93Mi            
flux-system           kustomize-controller-7b7b47f459-9qlvs              24m          43Mi            
flux-system           notification-controller-5bb6647999-4s28v           1m           16Mi            
flux-system           source-controller-7667765cd7-gpz2h                 2m           40Mi            
jupyterhub            continuous-image-puller-lfptt                      0m           0Mi             
jupyterhub            hub-8676597487-b2kkf                               2m           141Mi           
jupyterhub            proxy-7b7b858f6b-xpzsk                             1m           19Mi            
jupyterhub            user-scheduler-648565cdf7-2vb7f                    2m           23Mi            
jupyterhub            user-scheduler-648565cdf7-6qb6j                    3m           29Mi            
keda                  keda-admission-webhooks-6987d68f4c-sd4gh           1m           9Mi             
keda                  keda-operator-5c77476f9b-8vldr                     1m           32Mi            
keda                  keda-operator-metrics-apiserver-7c4675769c-rf5jg   3m           47Mi            
kube-system           calico-kube-controllers-77bd7c5b-gvr2r             1m           25Mi            
kube-system           calico-node-dxcht                                  19m          145Mi           
kube-system           coredns-6d896984d7-svbz4                           2m           23Mi            
kube-system           csi-secrets-store-secrets-store-csi-driver-s5rdz   1m           39Mi            
kube-system           hostpath-provisioner-7df77bc496-g2wl8              1m           17Mi            
metallb-system        controller-5f7bb57799-lmk68                        1m           36Mi            
metallb-system        speaker-m4wwb                                      2m           28Mi            
metrics-server        metrics-server-5dc9dbbd5b-b276n                    2m           32Mi            
nginx                 ingress-nginx-controller-59cf569798-6pmtg          4m           114Mi           
nginx                 ingress-nginx-controller-59cf569798-wv8nz          1m           99Mi            
ohdsi                 ohdsi-atlas-7cd45d4565-5w9rn                       0m           4Mi             
ohdsi                 ohdsi-hades-6f798d67bb-scjlc                       1m           75Mi            
ohdsi                 ohdsi-webapi-76f45574d8-8b2k8                      1m           1168Mi          
secrets-distributor   secrets-distributor-669864d94d-7d62t               0m           64Mi            

I've seen on other threads a recommendation for at least 2GB and I've seen other recommendations suggesting an allocation of up to 16GB of memory to be allocated to the environment which is fine in itself but if the service is using 1GB when idle then I worry how much this could expand when in use, with a single user logged in and using the base source it got up to 1937Mi.

Is there any way in which we could reduce the memory requirements of the service (especially when idle), or offload this requirement onto other services? Are there any magic settings which might help?

And what would the recommendation for right-sizing this service for use at scale in a production environment with multiple data sources?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the idle and single-user memory measurements for the ohdsi-webapi pod in Kubernetes, then inspect the WebAPI runtime and available configuration. Compare the observed usage with Hades and Atlas, and determine whether a concrete setting or service boundary can reduce the requirement. Done means documenting a supported right-sizing recommendation and any validated reduction path.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, kubernetes
Domain
backend, performance
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.