Altinn / Altinn/app-frontend-react
Record user interactions, form state and request/responses to generate automatic tests
- Dominant language
- TypeScript
- Stars
- 21
- Forks
- 33
- Avg merge
- 6d 23h
- Merged PRs (30d)
- 3
Description
### Background
Yet another issue that's been growing in the back of my head for a while. When delivering a platform for application and form development, it's important to decide who gets the responsibility of maintaining an app/form after deployment. Right now, we try our best to take responsibility for delivered frontend features via automatic browser-based testing (via Cypress). Sometimes we make mistakes, and end up releasing features or changes into production in a way that breaks existing applications because real production apps sometimes use obscure features, exploit side-effects of our previous implementation (and side-effects are features too), or probably most often, just use features we never got around to implement automatic tests for (and so we just never tested that scenario).
At some point, pressing on and delivering a constant stream of new features in app-frontend will just steadily grow the gap between which we have automatic tests for, and which features are in actual use in real-world apps. That's because some features can be combined, and implementing tests for every possible combination is hard.
And at the same time, we should strive towards providing value to our customers by creating a platform they trust, where they can develop their apps and forms, and hand them over to us so we can take care of deploying them, and increasingly also _maintain_ them for our customers. In some ways, our added value might even disappear if customers need to have developers on standby forever after developing their services on our platform. Apps should just work™.
### Solution
I propose a solution for this problem by implementing simple automatic testing targeted at our current abstraction level; components, layout, pages and form-filling. If we could capture what a user does in a way that allows us to create automatic Cypress tests (or similar), along with the expectations that go along with it, we would have an automated method of verifying that an app _still works as intended by the app developers_. That way, we could test out new releases (of both app-frontend and our backend) through the eyes and hands of virtual users.
By user actions, I mean things like:
- Typing in an `Input`
- Clicking on a checkbox
- Selecting something in a dropdown
- Adding a new row/deleting a row/clicking to edit a row in a repeating group
- Uploading an attachment
- Navigating to the next page
- Clicking a button
And by expectations I mean all the things the user doesn't do, but they see as an effect, like:
- The data model updates with a new value
- New components (child components an repeating group, for example) are now visible on the page
- Existing components are suddenly hidden
- When navigating to a new page, an entirely new set of components (with their values) are now visible
- When the entire form is submitted, the entire data model looks like
### Suggestions towards an implementation
This will be difficult to implement, but I believe it will provide a lot of value. My current thinking is that we should try to implement a layer that proxies all user actions that are supported in app-frontend. That means, for example for a repeating `Group` component, you would have to implement a `ClickAddNewButton` action. When it is triggered, a new row is added. When the user clicks the button themselves, we record that a `ClickAddNewButton` took place for that `Group`. This should preferably be implemented in a way that prevents developers from adding new actions to layout components without making sure to proxy them through this event system. This way, recording is just a matter of storing the list of actions that have been triggered, along with the effects it had.
Next, we could generate Cypress tests from a list of actions and effects. It would be nice to more tighly integrating our layout components with Cypress, such that we also could _write our own cypress tests_ in a way where we tell cypress to trigger a `ClickAddNewButton` for a given `Group`, and then the `Group` component itself could make sure to translate that request into actually fetching the DOM button and making Cypress click it.
I imagine a recorded actions log to look somewhat like this:
```json
[
{
"type": "effect",
"effect": "AppOpened",
"meta": {
"currentPage": "page1",
"visibleComponents": ["p1-input", "p1-next"],
"dataModel": {}
}
},
{
"type": "action",
"component": "p1-input",
"action": "type",
"meta": {
"text": "Ole Martin"
}
},
{
"type": "effect",
"effect": "dataModelChange",
"meta": {
"change": {
"FirstName": "Ole Martin"
},
"previous": {
"FirstName": null
}
}
},
{
"type": "action",
"component": "p1-next",
"action": "click"
},
{
"type": "effect",
"effect": "PageNavigation",
"meta": {
"currentPage": "page2",
"visibleComponents": ["p2-summary", "p2-submit"]
}
}
]
```
### Optimizations and further improvements
**Half-runs**
If we also record all requests/responses between the frontend and the backend during this process, we could use those to mock out either the frontend or the backend as well (ie. just sending requests back and forth to the backend when testing a new backend version, or mocking backend responses with these expected/prepared responses from the log to make frontend testing quicker).
The requests/responses, if added to the log, should probably not have the weight of other effects, as we'll probably want to change the contracts between frontend and backend periodically. In those cases, we might want to automatically re-create new test logs from "full" runs with both the frontend and backend.
**Browserless**
With an interface to an app being represented through these actions and effects, we could probably emulate the entire app without actually executing the code in a browser and rendering it to a screen. In fact, the rendering is probably a very expensive part. For lighter and faster test-runs we could run the tests on app-frontend (and possibly simulate the backend using the expected responses from above) in a browserless environment, by simply executing the actions in the context of the app and waiting for the expected effects to be triggered.
**Visual testing**
As noted in one of the linked related issues, there are tools to look at an app visually (via screenshots) and compare before/after changes. We could use such tooling, if valuable and not too expensive, to compare the visuals before/after a planned release of app-frontend to catch visual regressions.
### Related issue(s)
- https://github.com/Altinn/app-template-dotnet/issues/122
- #1021
- #952
Contributor guide
Research direction
No implementation files, tests, or entry points are named. Start by surveying the app-frontend-react action handling and existing Cypress integration, then define a focused first slice of the proposed recording system; done should include a documented scope, captured user actions and effects, and generated tests that verify the recorded behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cypress, react, typescript
- Domain
- frontend, testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100