game-ci / game-ci/unity-test-runner

Feature: Add Caching for Package Testing

Open
#226 1 comment 2 reactions 0 assignees View on GitHub
enhancement
Dominant language
TypeScript
Stars
265
Forks
149
Avg merge
3h 3m
Merged PRs (30d)
2

Description

**Context**

Package testing is done by the following series of steps:
1. creating a Unity Project
2. Adding the Unity Package being tested to that project
3. Running tests on the Unity Project.

It would be nice to somehow cache the Unity Project after the first run of the action so that it doesn't have to be remade each time the action is run.

**Suggested solution**

Leveraging [GitHub's existing cache library on NPM](https://www.npmjs.com/package/@actions/cache) seems like it would be the right way to go here.

I haven't really worked with the feature itself in a while, so this may have some mistakes, but I think that the following **non-exhaustive** list of changes would be a good start for a solution to the problem:

1. In `src/main.ts`, use `@actions/cache` to check/restore the cache for the Unity Project which the test runner creates.
2. In `src/main.ts`, add an argument to `Docker.run` called `cachedProjectPath` or something like that, which will be blank if the cache was not found, and the path to the Unity Project which was restored from the cache if the cache was found. Ensure that the new argument is set up properly to be available to the bash scripts.
3. In `dist/run_tests.sh`, wrap the creation of the Unity Project (the `unity-editor -batchmode -createProject` call) in a conditional that checks for the `cachedProjectPath` variable. If it's blank, do what we do now - create a new Unity project and add the package to the project's dependencies. If the cached project path is present, re-add the package to the cached project's dependencies (or otherwise check that the project is depending on the package correctly).
4. Have the tests run as normal
5. If the cache wasn't found, cache the Unity project that was newly created.

I say that those steps are non-exhaustive, as it ignores certain implementation details, **included but not limited to**:
- How the cache key is set up for hits and misses. We should certainly take Unity version into account here, but there may be more considerations past that.

**Considered alternatives**

An alternative could be to somehow factor out the creation of the Unity Project (step 1 in the context section above) to its own workflow step before the test runner runs. Then, that project could be independently cached with the [cache action](https://github.com/actions/cache), and somehow passed in to the test runner.

Despite the fact that this kind of gives more control to the user, I think that this would be a worse solution though, as it still requires a good bit of work for way worse UX for the average user.

**Additional details**

There has been at least two people who have requested this feature, [as can be seen here](https://github.com/game-ci/unity-test-runner/issues/71#issue-709463698) and [here](https://github.com/game-ci/unity-test-runner/issues/71#issuecomment-1126947149).

Contributor guide

Open the contributing guide

Research direction

Start by reading src/main.ts, especially the test-runner flow and Docker.run arguments, then inspect dist/run_tests.sh around Unity project creation and package setup. Use @actions/cache documentation to define cache hit and miss behavior, including Unity version in the key; done means tests run with either a restored or newly created project and only cache misses save the project.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, typescript, unity
Domain
ci-cd, testing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.