Allow building of tests only for specific module

Currently, when running cmake with -DBUILD_TESTS=ON, it causes all compile units to build tests, including projects that you e.g. fetchcontent into your project.
This is undesirable, because when running `make` all dependency tests are compiled, and when running `make test` it runs all tests of dependencies.

Is it possible to specify for which compile unit to build the tests?
Or perhaps there is a way to only enable tests for compile units that were added in the current CMakeLists.txt,
perhaps with a different value for -DBUILD_TESTS=LOCAL or something?