CatsResourceIO in scalatest doesn't release the resource

Info:

cats-effect-testing-scalatest" % "1.4.0" 
openjdk 11.0.12

Given following example:

import cats.effect._
import cats.effect.testing.scalatest.{AsyncIOSpec, CatsResourceIO}
import org.scalatest.freespec.FixtureAsyncFreeSpec

class TestExample extends FixtureAsyncFreeSpec with AsyncIOSpec with CatsResourceIO[Int] {
  val resource: Resource[IO, Int] = Resource.make(IO.println("acquired").as(1))(_ => IO.println("released"))

  "should close resource" in { outer =>
    val inner = Resource.make(IO.println("acquired inner"))(_ => IO.println("released inner"))
    inner
      .use { _ =>
        IO.println(outer).as(succeed)
      }
  }
}

I would expect scalatest to release the fixture resource. Instead the resource is not being released.

I think that there are two problems with the current solution.

First, we schedule the shutdown action here: https://github.com/typelevel/cats-effect-testing/blob/series/1.x/scalatest/shared/src/main/scala/cats/effect/testing/scalatest/CatsResource.scala#L76 which is asynchronous and then we immediately clear the shutdown variable. This clear action has a potential to be executed before the scheduled future is run resulting in the release code not being evaluated.

However, only rewriting that to something like:

  override def afterAll(): Unit = {
    UnsafeRun[F].unsafeToFuture(
      shutdown >> Sync[F] delay {
        gate = None
        value = None
        shutdown = ().pure[F]
      },
      finiteResourceTimeout
    )
  }

Doesn't fix the problem.

The second part is somehow related to the execution context used by scalatest.

I didn't dig deep enough into the scalatest codebase to prove that but it seems to me that the execution context gets closed(?) as soon as the test suite completes.

Changing the EC used to evaluate future actions together with previous solution fixes the problem.

  private lazy val _ResourceUnsafeRun =
    new UnsafeRun[IO] {
      private implicit val runtime: IORuntime = createIORuntime(ExecutionContext.global) //here using global instead of inherited exectionContext

      override def unsafeToFuture[B](ioa: IO[B]): Future[B] =
        unsafeToFuture(ioa, None)

      override def unsafeToFuture[B](ioa: IO[B], timeout: Option[FiniteDuration]): Future[B] =
        timeout.fold(ioa)(ioa.timeout).unsafeToFuture()
    }

I am not saying that this is the correct solution, I am only including that to give more information about the underlying issue.

While trying to understand that issue I also came across these two issues which I think might be relevant:

• https://github.com/scalatest/scalatest/issues/953 - TLDR: there is no beforeAllAsync and afterAllAsync for now
• https://github.com/scalatest/scalatest/issues/1828 - TLDR: you should not block within before and after methods as default ExecutionContext is single threaded

Start at scalatest/shared/src/main/scala/cats/effect/testing/scalatest/CatsResource.scala around the afterAll shutdown code, then reproduce the TestExample with the provided cats-effect-testing-scalatest and OpenJDK versions. Trace the shutdown future and execution context, and consider the issue complete when the fixture's release action reliably runs after the suite finishes.