ansys / ansys/pydynamicreporting
Add automation-friendly template JSON import and export results
- Dominant language
- Python
- Stars
- 12
- Forks
- 5
- Avg merge
- 3d 14h
- Merged PRs (30d)
- 9
Description
### 📝 Description of the feature
Make serverless template JSON import and export suitable for reliable automation while preserving existing template-file compatibility.
The current APIs successfully move a template tree through JSON, but their return and file-write contracts leave automation without enough information:
```python
def ADR.load_templates_from_file(self, file_path: str | Path) -> None:
...
self.load_templates(templates_json)
def ADR.load_templates(self, templates: dict) -> None:
...
root_template = self._populate_template(...)
root_template.save()
self._build_templates_from_parent(...)
def Template.to_json(self, filename: str) -> None:
...
if not filename.endswith(".json"):
filename += ".json"
...
os.chmod(filename, 0o444)
```
Current limitations:
- `load_templates_from_file()` and `load_templates()` return `None`, even though every imported template receives a newly generated GUID.
- Callers cannot directly identify the created root or map JSON IDs such as `Template_0` to persisted GUIDs.
- Import persists the root and descendants incrementally. If validation or saving fails on a later descendant, an incomplete tree can remain in the database.
- The loader selects the first root it encounters rather than explicitly rejecting documents with multiple roots.
- `Template.to_json()` returns `None`, so callers must reproduce the `.json` suffix rule to determine the actual path.
- Export always applies mode `0o444`, has no explicit read-only option, and has no documented overwrite policy.
- Export writes directly to the destination rather than atomically replacing it after serialization succeeds.
Specific historical JSON-type bugs are already covered by #244 and #279. This issue is about the general automation contract: structured results, atomicity, and explicit file behavior.
### 💡 Steps for implementing the feature
#### 1. Return a structured import result
Add a result type that exposes both product objects and JSON-to-database identity mapping:
```python
from dataclasses import dataclass
from typing import Mapping
@dataclass(frozen=True, slots=True)
class TemplateImportResult:
root: Template
templates: tuple[Template, ...]
source_id_to_guid: Mapping[str, str]
@property
def count(self) -> int:
return len(self.templates)
```
Update both import entry points:
```python
def load_templates_from_file(
self,
file_path: str | Path,
) -> TemplateImportResult:
...
def load_templates(self, templates: Mapping[str, object]) -> TemplateImportResult:
...
```
Returning a value is backward compatible for callers that currently ignore the result.
The mapping should use the JSON document's keys, for example:
```python
result.source_id_to_guid == {
"Template_0": "",
"Template_1": "",
}
```
Return templates in deterministic document/tree traversal order with the root first.
#### 2. Validate the complete document before persistence
Perform a structural pass before creating any database objects:
- require exactly one root;
- require every referenced child and parent ID to exist;
- require parent and child declarations to agree;
- reject cycles;
- reject disconnected/unreachable nodes;
- reject a child referenced by multiple parents;
- validate every template's required keys, report type, filter, params, and sort selection;
- include the source template ID in every validation error.
Represent the validated tree separately from persisted objects so validation has no database side effects.
#### 3. Make persistence transactional
Create the full tree inside one database transaction and collect results during recursion:
```python
from django.db import transaction
with transaction.atomic(using="default"):
created: list[Template] = []
source_id_to_guid: dict[str, str] = {}
root = self._create_validated_tree(
validated_document,
created=created,
source_id_to_guid=source_id_to_guid,
)
return TemplateImportResult(
root=root,
templates=tuple(created),
source_id_to_guid=MappingProxyType(dict(source_id_to_guid)),
)
```
Any validation, integrity, or save failure must roll back the whole imported tree. Do not leave a root or partial descendants behind.
#### 4. Return the actual export path and expose explicit policies
Extend `Template.to_json()` with keyword-only options whose defaults preserve the current intended behavior:
```python
def to_json(
self,
filename: str | Path,
*,
overwrite: bool = True,
read_only: bool = True,
) -> Path:
"""Atomically export this root tree and return the actual JSON path."""
```
Rules:
- preserve the current root-only restriction;
- preserve suffix appending when `.json` is absent;
- return the final `Path` after suffix normalization;
- if the destination exists and `overwrite=False`, raise `FileExistsError` before writing;
- if `read_only=False`, leave the new file writable according to normal platform defaults;
- if `read_only=True`, apply the read-only policy after the final file is in place;
- document behavior on POSIX and Windows rather than assuming `0o444` has identical semantics;
- never mutate permissions on an unrelated pre-existing file when export ultimately fails.
Returning `Path` and adding keyword-only options remain compatible with existing callers that pass only a filename and ignore the return value.
#### 5. Write exports atomically
Serialize to a temporary file in the destination directory and replace only after JSON serialization and flush succeed:
```python
target = normalize_json_path(filename)
temporary = target.with_name(f".{target.name}.{uuid.uuid4().hex}.tmp")
try:
with temporary.open("w", encoding="utf-8") as stream:
json.dump(self.to_dict(), stream, indent=4)
stream.flush()
os.fsync(stream.fileno())
if target.exists() and not overwrite:
raise FileExistsError(target)
os.replace(temporary, target)
if read_only:
target.chmod(0o444)
finally:
temporary.unlink(missing_ok=True)
return target
```
Handle replacement of an existing read-only destination explicitly on Windows. A failed export must preserve the previous destination contents and permissions.
#### 6. Add focused tests
Import tests:
1. result contains the created root, all created templates, and every JSON-ID/GUID pair;
2. return order follows tree order;
3. imported GUIDs are new and the mapping is immutable;
4. invalid late descendants roll back the root and all earlier descendants;
5. zero roots and multiple roots fail;
6. missing references, cycles, duplicate parents, and disconnected nodes fail before persistence;
7. file and dictionary entry points return equivalent results.
Export tests:
1. returned path includes an appended `.json` suffix;
2. `overwrite=True` replaces successfully and atomically;
3. `overwrite=False` preserves the existing file;
4. `read_only=True` and `read_only=False` have the documented platform behavior;
5. serialization or replacement failure preserves the previous file;
6. non-root export still raises the existing product exception;
7. a save/load round trip returns a discoverable root and complete ID/GUID mapping.
#### Acceptance criteria
- Import either persists one complete valid tree or persists nothing.
- Import returns the created root, deterministic created-object list, and source-ID/GUID mapping.
- Export returns the actual path after suffix handling.
- Overwrite and read-only behavior are explicit keyword options.
- Export replacement is atomic and failure-safe.
- Existing one-argument `to_json()` and result-ignoring import callers remain compatible.
- Documentation contains an end-to-end automation example that needs no follow-up query by template name.
### 🔗 Useful links and references
- [Current `load_templates_from_file()` and incremental import](https://github.com/ansys/pydynamicreporting/blob/d98b45adad663d015d4e17ad93dcd3749f2f3d81/src/ansys/dynamicreporting/core/serverless/adr.py#L1043-L1118)
- [Current `Template.to_json()` behavior](https://github.com/ansys/pydynamicreporting/blob/d98b45adad663d015d4e17ad93dcd3749f2f3d81/src/ansys/dynamicreporting/core/serverless/template.py#L513-L535)
- Related but non-duplicate issues: #244 and #279 fixed particular template-type import failures rather than the general result/transaction/file-policy contract.
An open-and-closed issue audit found no existing issue requesting structured import results or explicit atomic template-file export behavior.
Contributor guide
Assessment
This issue has not been assessed yet.