intersystems / intersystems/ipm

IPM needs a structured logging framework for (at the very least) module deployment

Open
#1,240 0 comments 0 reactions 1 assignee Claimed by @isc-eneil View on GitHub
enhancement
Dominant language
ObjectScript
Stars
41
Forks
29
Avg merge
23h 54m
Merged PRs (30d)
4

Description

## Link to the discussion thread
https://github.com/intersystems/ipm/discussions/1239

## Describe the problem
Currently, IPM logging consists of writing to standard output, which is redirected to the terminal when using the zpm shell. IPM also now has the history command which holds onto a timestamped trace of all IPM commands run in an ipm-enabled namespace. There is also the possibility of logging to a file via the %IPM.Main::ShellScript() method.

As @AshokThangavel noted in a few issues that he created (https://github.com/intersystems/ipm/issues/1048 https://github.com/intersystems/ipm/issues/1012), as IPM's functionality expands to include installation and updating of packages in production environments, it needs a built-in framework for structured logging to a persistent file that can be inspected.

## Describe the solution

### Overview
Extend the %IPM.General.LogManager class to support writing output to an unstructured log file intended for humans and a structured, AI-readable JSON log file. All log lines will be logged to both log files, in a different format. Logging to these two log files will occur by default on the `install` , `update` , and `uninstall` commands (to start) and can be turned off using the `-no-log` modifier. The log file path can be changed via the `-output-file` modifier. Log files for each of these command will be linked to the corresponding history log entry (used for auditing). Documentation will be added to make IPM logging APIs clear to developers of module update steps.

### User Flow

1. Run update -v myModule 2.1.1 inside of MODULE_NS namespace
2. Two logs will be created:
- /mgr/ipm-logs/MODULE_NS/update_myModule_1787775895.json
- /mgr/ipm-logs/MODULE_NS/update_myModule_1787775895.txt

update_myModule_1787775895.json will look something like:
```
{
"ipm": 0.10.9,
"schema": 1.0.0,
"command" : "update -v myModule 2.1.1",
"log" : [
{ "level": "INFO", "message": "Running zpm \"update -v myModule 2.1.1\"", "timestamp": "1787775895"},
{ "level": "INFO", "message": "Preparing to update module myModule to version 2.1.1", "timestamp": "1787775909"},
{ "level": "INFO", "message": "Loading from /home/irisowner/zpm/tests/integration_tests/Test/PM/Integration/_data/update-test/myModule-2.x/myModule/", "timestamp": "1787775910"},
{ "level": "INFO", "message": "Load started on 07/09/2026 16:14:58", "timestamp": "1787775911"},
{ "level": "INFO", "message": "Loading file /home/irisowner/zpm/tests/integration_tests/Test/PM/Integration/_data/update-test/myModule-2.x/myModule/module.xml as xml", "timestamp": "1787775912"},
{ "level": "INFO", "message": "Imported document: myModule.ZPM", "timestamp": "1787775913"},
{ "level": "INFO", "message": "Load finished successfully.", "timestamp": "1787775913"},
{ "level": "INFO", "message": "Building dependency graph [START]", "timestamp": "1787775913"},
{ "level": "ERROR", "message": "ERROR 5001: Cannot find module.xml for module MyModule.", "timestamp": "1787775913"},
{"level": "INFO", "message": "Logging COMPLETE", "timestamp": "1787775914"}
]
}
```
update_myModule_1787775895.txt will look something like:
```
IPM Version: 0.10.9
Schema Version: 1.0.0
Command: "update -v myModule 2.1.1"
[INFO] Running zpm "update -v myModule 2.1.1"
[INFO] Preparing to update module myModule to version 2.1.1
[INFO] Loading from /home/irisowner/zpm/tests/integration_tests/Test/PM/Integration/_data/update-test/myModule-2.x/myModule/
[INFO] Load started on 07/09/2026 16:14:58
[INFO] Loading file /home/irisowner/zpm/tests/integration_tests/Test/PM/Integration/_data/update-test/myModule-2.x/myModule/module.xml as xml
[INFO] Imported document: myModule.ZPM
[INFO] Load finished successfully.
[INFO] Building dependency graph [START]
[ERROR] ERROR 5001: Cannot find module.xml for module MyModule.
[INFO] Logging COMPLETE
```
3. To debug a failed command, pass the AI-readable log to an LLM. A SKILLS.md file provided by ISC module developers may offer the model with guidance on troubleshooting updates/installations.
4. This may point you to inspect a particular part of the human-readable logs (providing a particular log line to search for)

### Log Format
- Log files will be stored under `/mgr/ipm-logs/` by default. Configurable via `-output-file ` modifier.
- Naming convention: __-, e.g. install_myModule_1787775895-1
- All log files will begin with metadata (e.g. IPM version, schema version, command run to generate the log with credentials obfuscated) and end with a line indicating logging completion.

#### Human-readable Text File
- .txt file with trace, error, warning, and info output.
- Each line will be prefixed with either [TRACE], [INFO], [ERROR], or [WARNING]

#### AI-readable JSON File
- .json file
- JSON array with log metadata followed by one JSON object per log line that looks as follows:

```
{
"ipm": ,
"schema": ,
"command" : ,
"log" : [
{
"level": <"INFO", "ERROR", or "WARNING>
"message": ,
"timestamp":
}
]
}
```
- Output redirected from IRIS may include multiple lines. Where possible, log such "chunks" of output as single "message" values with "\n" signifying newlines.

## Describe alternatives
A few additional layers of complexity that didn't make it into the final design:
- Considered a more complex structured logging approach with logging event "types" such as Terraform uses, but decided that this was overkill (need to test if this even improves filtering or diagnosing once we've implemented this)
- Considered a more complex multi-level logging structure such as npm has (with silly, debug, http, trace, etc) that allows users granular filtering control over log contents, but in speaking with internal IPM deployment stakeholders, I determined that none were interested in less verbosity within logs. Instead, I propose a simple trace, info, error, warning categorization that is simply used to help sort log lines but not filter them out of the file.

## Additional context
This was inspired by Terraform's machine readable logging framework (https://developer.hashicorp.com/terraform/internals/machine-readable-ui).

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.