Replace ClassGraph classpath scanning with ServiceLoader or something else

What problem are you trying to solve?

org.openrewrite.config.Environment.Builder#scanClassLoader causes OutOfMemoryError

Caused by: io.github.classgraph.ClassGraphException: Uncaught exception during scan
    at io.github.classgraph.ClassGraph.scan(ClassGraph.java:1637)
    at io.github.classgraph.ClassGraph.scan(ClassGraph.java:1654)
    at io.github.classgraph.ClassGraph.scan(ClassGraph.java:1667)
    at org.openrewrite.config.ClasspathScanningLoader.scanClasses(ClasspathScanningLoader.java:143)
    at org.openrewrite.config.ClasspathScanningLoader.<init>(ClasspathScanningLoader.java:74)
    at org.openrewrite.config.Environment$Builder.scanClassLoader(Environment.java:229)
Caused by: java.lang.OutOfMemoryError: Java heap space

It is hard to display in heap dump analysis, however, scanning Apache JMeter classpath triggers OOM with -Xmx2500m heap.

Describe the solution you'd like

OpenRewrite should either load recipes on demand or it should use ServiceLoader to locate services.
It would avoid spending time on classpath scan.

Have you considered any alternatives or workarounds?

Can OpenRewrite search the recipe when user requests the exact recipe name?

For instance:

Caused by: org.openrewrite.RecipeException: Recipes not found: org.openrewrite.java.testing.junit5.JUnit5BestPractices, org.openrewrite.java.testing.junit5.CleanupAssertions
    at org.openrewrite.config.Environment.activateRecipes(Environment.java:154)

Why throwing Recipes not found exception? Can it just request Class.forName("org.openrewrite.java.testing.junit5.JUnit5BestPractices") and or classLoader.getResources("META-INF/openrewrite/recipes.yml")? Then it won't need to scan all the classpath jars for every execution.

An alternative option could be adding https://github.com/smallrye/jandex annotation index, however, it would be an overkill.

Additional context

I am integrating OpenRewrite to Apache JMeter, and having OR trigger OOM is painful: https://github.com/apache/jmeter/pull/6217

Start with org.openrewrite.config.Environment.Builder#scanClassLoader and ClasspathScanningLoader#scanClasses, then reproduce the failure using the Apache JMeter classpath and the reported -Xmx2500m limit. Compare the requested recipe lookup behavior with the current classpath scanning, and consider the work done when recipe discovery no longer scans the full classpath or triggers the reported OutOfMemoryError.