Project-MONAI / Project-MONAI/monai-deploy-app-sdk
[FEA] Provide tracing and performance statistics gathering capability in App SDK
@MMelQin is already working on this.
Since Jun 23, 2023.
- Dominant language
- Python
- Stars
- 138
- Forks
- 70
- PR merge metrics
- No merged PRs in 30d
Description
Is your feature request related to a problem? Please describe.
With the use of logging library, application execution logs can be gathered at various logging level per configuration. However, with the maturing of the App SDK, especially when venturing into clinical deployment (IRB'ed), there are stronger needs to provide additional insights into the application execution, e.g. activity audits, performance metrics both in terms of latency and ideally result accuracy.
This is important when AI applications are deployed and used in imaging workflow, as the IT/PACS admins always need to have insights into the latency, resource, and clinical performance of the applications, so as to better manage the critical workflow stage for operational, clinical, as well as regulatory requirements.
Describe the solution you'd like
A logging framework that will support pre-defined as well as custom categories of metrics collection, while making the aggregation and delivery of the metrics dynamic and configurable, e.g. records gather could be delivered to the stats gathering agent on the host system that runs the MAP.
Describe alternatives you've considered
Native logging library with logging level combined with defined string literal indicating the categories of information, e.g. default being execution log, "PERF" for elapsed time for blocks of execution, etc.
Additional context
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.