KSP plugin works fine when compiling with gradle but fails when compiling with Intellij IDEA

The very simple project here demonstrates the issue:

https://github.com/aduchate/kmap-test

When compiled using gradle, everything goes fine but when compiled using IntelliJ, the following error occurs

```
Kotlin: [Internal Error] java.lang.IllegalStateException: The provided plugin com.google.devtools.ksp.KotlinSymbolProcessingComponentRegistrar is not compatible with this version of compiler
at org.jetbrains.kotlin.cli.jvm.compiler.KotlinCoreEnvironment$Companion.registerExtensionsFromPlugins$cli(KotlinCoreEnvironment.kt:667)
at org.jetbrains.kotlin.cli.jvm.compiler.KotlinCoreEnvironment$ProjectEnvironment.registerExtensionsFromPlugins(KotlinCoreEnvironment.kt:169)
...
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.AbstractMethodError: Receiver class com.google.devtools.ksp.KotlinSymbolProcessingComponentRegistrar does not define or inherit an implementation of the resolved method abstract registerProjectComponents(Lcom/intellij/mock/MockProject;Lorg/jetbrains/kotlin/config/CompilerConfiguration;)V of interface org.jetbrains.kotlin.compiler.plugin.ComponentRegistrar.
at org.jetbrains.kotlin.cli.jvm.compiler.KotlinCoreEnvironment$Companion.registerExtensionsFromPlugins$cli(KotlinCoreEnvironment.kt:659)
... 28 more
```

I'm using IntelliJ iDEA 2022.01, Kotlin 1.6.20 and KSP 1.6.21-1.0.5

I checked in the source code of KSP and registerProjectComponents has the right signature.
Any idea why this issue might arise or any way to disable the plugin when the code is compiled by IntelliJ ?

Thanks in advance