finos / finos/architecture-as-code

Add snapshots feature to CalmHub and tooling to support scratch work

Open
#3,023 0 comments 0 reactions 1 assignee Claimed by @willosborne View on GitHub
Roadmap Group: CALM Tools Roadmap: Next
Dominant language
TypeScript
Stars
399
Forks
138
Avg merge
2d 6h
Merged PRs (30d)
38

Description

## Feature Proposal

### Target Project:
CalmHub, CLI

### Description of Feature:
When working on pattterns, architectures etc, to be published to CalmHub, it's useful to be able to publish them to visualise + work with them in the UI.
However, these patterns usually need multiple iterations as experimental work.

This issue proposes the introduction of a 'snapshots' feature, similar to Maven's appraoch to this.

When publishing a document, you can add `-SNAPSHOT` to the end of its version to mark it as a SNAPSHOT. Only one snapshot can be associated with a given semantic version.

Snapshots are **mutable** - pushing again to the snapshot will **overwrite it** in the database.
Then when work is finished, the user can publish the full version without the -SNAPSHOT suffix; this will delete the snapshot.

### User Stories:
- As an architect, I want a snapshot of a document, so I can quickly iterate on patterns and architectures.

### Current Limitations:
- Currently there are only immutable versions.
- There is no concept of promotion between namespaces or CalmHub instances.

### Proposed Implementation:
When creating a resource via the `POST` or `PUT /calm/**` user-facing endpoints, put `-SNAPSHOT` into the version. (Note: POST creates, you can only update a snapshot with PUT.)

For example:

```bash
POST /calm/namespaces/workshop/architectures/test/versions/1.0.0-SNAPSHOT
201 Created

# then to update:
PUT /calm/namespaces/workshop/architectures/test/versions/1.0.0-SNAPSHOT
204 No Content

# … many changes later, create the final version without the suffix:
POST /calm/namespaces/workshop/architectures/test/versions/1.0.0
201 Created

# this DELETES the snapshot:
GET /calm/namespaces/workshop/architectures/test/versions/1.0.0-SNAPSHOT
404 Not Found
```

#### UI changes

Snapshots will appear in the UI stamped with a snapshot label. They will appear on timelines too, so you can see the diff to the current snapshot.

#### Tooling support

This would then be added to the `hub` commands via a `--snapshot` flag:

`calm hub push architecture --namespace abcd --mapping test --version 1.0.1 --snapshot`

...and the workspace commands by adding a `--snapshot` flag to `bump` and `push`.

When working on a document, if you want to move to a snapshot version, run `calm workspace bump --snapshot`.
This will bump any modified files as usual, except the NEW version will be the snapshot.
No files will be pushed.

Then to restore something from a snapshot i.e. to publish the current state:

`calm workspace bump --publish-snapshots`

This will update the version of all snapshots to no longer have the `-SNAPSHOT` suffix, and update any dependent documents in the workspace to the final version.

#### BREAKING CHANGE: default `calm workspace push` behaviour

In this model I believe it will be very easy to accidentally publish to prod when you meant to just do a snapshot change.
This is because snapshots as well as actual writes are both gated behind the WRITE permission.

There are two options here:
- introduce `WRITE_SNAPSHOT` as a permission that works exactly like `WRITE` except you can ONLY push snapshots
- change the default auto-bump behaviour of the command.

I suggest the latter as I would like to keep the permissions model as simple as possible; but would like input on this decision.

My proposal is to just completely remove the auto-bump logic from `push` as when working with a PROD environment it's easy to make a mistake.
After this change, `calm workspace push` will **NEVER** automatically bump the version of changed files.

It will defensively run `check` first and report an error if anything has changed but not been bumped.
However, for snapshots this is **NOT** an issue: since snapshots are mutable, push can push them just fine without a bump.

#### Final workspace workflow:

- Make some changes
- `calm workspace bump --snapshot`: bump any changed files to snapshots.
- `calm workspace push`: publish the snapshots
- Make further changes as required
- `calm workspace push`: re-publish any changed snapshots. **If you change a new file that wasn't marked as snapshot, this will fail.**
- When you're happy, run `calm workspace bump --publish-snapshots`
- ...and finally `calm workspace push` to finally publish your documents.

Note that if you do not specify --snapshot on the bump command, you won't use the snapshots feature at all. i.e. default workflow is the same.
However this means you will need to bump after every push as the versions will NOT be mutable.

### Alternatives Considered:
An 'environments' system where each workspace can have named environment labels targeting different CalmHubs, different namespaces, etc. Then a `calm workspace migrate` command that moves a document into a new environment, changing the IDs and so on as appropriate.

This was quite complicated and we decided snapshots was a simpler solution.

### Testing Strategy:
Unit and integration tests for the CLI and CalmHub

### Documentation Requirements:
CLI docs, calmhub docs, user guide on the main site

### Implementation Checklist:
- [ ] Design reviewed and approved
- [ ] Implementation completed
- [ ] Tests written and passing
- [ ] Documentation updated
- [ ] Relevant workflows updated (if needed)
- [ ] Performance impact assessed

### Additional Context:
Questions raised in this issue:
- **Should snapshots only be allowed on the latest version?**
- **Should we split the `write` permission to prevent accidental pushes?**
- **Are people happy with removing auto-bump from the `push` command?**

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.