typelevel / typelevel/cats-effect
Laws testing with Arbitrary[IO[A]] from TestInstances
Nobody has claimed this yet.
- Dominant language
- Scala
- Stars
- 2.2k
- Forks
- 576
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 18
Description
I've been recently playing around with migration to CE3. Some of my libraries provide algebras that are law tested. Some of them are meant to be used against IO, thus the laws were also tested against it. In CE2 I would use cats.effect.laws.discipline.arbitrary._ and as discussed on Discord the right replacement would be to have my test suite extend TestInstances and instantiate implicit Ticker().
With this approach my migrated tests wouldn't work out well, but I've learned that Parallel is a tricky one to law test, so I've eliminated that one from my tests.
Despite that effort, I still face issues with law testing quite simple algebras' properties. I've prepared a repository reproducing my issues: https://github.com/majk-p/io-laws-playground/
For the demo I've created two simple algebras, here are the simplified descriptions
trait Storage[F[_], A] {
def store(a: A): F[Unit]
}
and
trait Consumer[F[_], +A] {
def consume(f: A => F[Unit]): F[Unit]
}
Say I want to test that the laws of Monoid hold for those two. I've prepared a test suite in:
https://github.com/majk-p/io-laws-playground/blob/master/src/test/scala/net/michalp/StorageLaws.scala
While the tests in DummyStorageLaws (that use IO.pure as ArbitraryIO implementation) and SyncIOStorageLaws (that use TestInstances provided Arbitrary[SyncIO[A]]) or even just OptionLawSpec work perfectly well, implementation based on Arbitrary[IO[A]] from TestInstances fails. The error is more or less like the one below for most tests
[info] IOStorageLaws
[info] monoid laws must hold for Monoid[Storage[IO, Int]]
[error] ! monoid.associative
[error] java.lang.UnsupportedOperationException: Exception raised on property evaluation.> ARG_0: <funct
ion1>
[error] > ARG_1: <function1>
[error] > ARG_2: <function1>> Exception: java.lang.UnsupportedOperationException: empty.min
[error] The seed is 2ZuVq_ssmSpdX6GM8VKNsiFWtPudTuIZT-WgWQIafPP= (TestContext.scala:98)
[error] cats.effect.kernel.testkit.TestContext.nextInterval(TestContext.scala:98)
[error] cats.effect.kernel.testkit.TestContext.tickAll(TestContext.scala:179)
[error] cats.effect.testkit.TestInstances.unsafeRun(TestInstances.scala:193)
[error] cats.effect.testkit.TestInstances.unsafeRun$(TestInstances.scala:183)
[error] net.michalp.IOStorageLaws.unsafeRun(StorageLaws.scala:23)
[error] cats.effect.testkit.TestInstances.$anonfun$eqIOA$1(TestInstances.scala:145)
[error] cats.kernel.Eq$$anon$2.eqv(Eq.scala:71)
[error] cats.laws.discipline.eq$.$anonfun$catsLawsEqForFn1Exhaustive$2(Eq.scala:17)
[error] cats.laws.discipline.eq$.$anonfun$catsLawsEqForFn1Exhaustive$2$adapted(Eq.scala:17)
[error] cats.laws.discipline.eq$.$anonfun$catsLawsEqForFn1Exhaustive$1(Eq.scala:17)
[error] cats.laws.discipline.eq$.$anonfun$catsLawsEqForFn1Exhaustive$1$adapted(Eq.scala:17)
[error] cats.kernel.Eq$$anon$5.eqv(Eq.scala:97)
[error] cats.kernel.laws.discipline.package$.catsLawsIsEqToProp(package.scala:11)
[error] cats.kernel.laws.discipline.SemigroupTests.$anonfun$semigroup$2(SemigroupTests.scala:18)
[error] org.scalacheck.Prop$.$anonfun$forAllShrink$2(Prop.scala:768)
[error] org.scalacheck.Prop$.secure(Prop.scala:478)
[error] org.scalacheck.Prop$.result$1(Prop.scala:768)
[error] org.scalacheck.Prop$.$anonfun$forAllShrink$1(Prop.scala:807)
[error] org.scalacheck.Prop$.$anonfun$apply$1(Prop.scala:308)
[error] org.scalacheck.PropFromFun.apply(Prop.scala:21)
[error] org.scalacheck.Prop$.result$1(Prop.scala:769)
[error] org.scalacheck.Prop$.$anonfun$forAllShrink$1(Prop.scala:807)
[error] org.scalacheck.Prop$.$anonfun$apply$1(Prop.scala:308)
[error] org.scalacheck.PropFromFun.apply(Prop.scala:21)
[error] org.scalacheck.Prop$.result$1(Prop.scala:769)
[error] org.scalacheck.Prop$.$anonfun$forAllShrink$1(Prop.scala:807)
[error] org.scalacheck.Prop$.$anonfun$apply$1(Prop.scala:308)
[error] org.scalacheck.PropFromFun.apply(Prop.scala:21)
[error] org.scalacheck.Prop$.$anonfun$delay$1(Prop.scala:483)
[error] org.scalacheck.Prop$.$anonfun$apply$1(Prop.scala:308)
[error] org.scalacheck.PropFromFun.apply(Prop.scala:21)
[error] org.scalacheck.Prop.$anonfun$viewSeed$1(Prop.scala:40)
[error] org.scalacheck.Prop$.$anonfun$apply$1(Prop.scala:308)
[error] org.scalacheck.PropFromFun.apply(Prop.scala:21)
[error] org.scalacheck.Prop$.$anonfun$apply$1(Prop.scala:308)
[error] org.scalacheck.PropFromFun.apply(Prop.scala:21)
[error] org.scalacheck.Test$.workerFun$1(Test.scala:434)
[error] org.scalacheck.Test$.$anonfun$check$2(Test.scala:466)
[error] org.scalacheck.Test$.$anonfun$check$2$adapted(Test.scala:466)
[error] org.scalacheck.Platform$.runWorkers(Platform.scala:40)
[error] org.scalacheck.Test$.check(Test.scala:466)
[error] net.michalp.IOStorageLaws.check(StorageLaws.scala:23)
[error] org.typelevel.discipline.specs2.mutable.Discipline.$anonfun$checkAll$2(Discipline.scala:37)
Please advise if I'm doing something wrong when using library provided arbitraryIO, or is it really a bug?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the failure using the linked io-laws-playground example, starting with src/test/scala/net/michalp/StorageLaws.scala. Trace the stack through cats.effect.kernel.testkit.TestContext.nextInterval and cats.effect.testkit.TestInstances. Done means determining whether Arbitrary[IO[A]] from TestInstances is being used incorrectly or exposes a library bug, with the behavior documented or corrected.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- scala
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100