typelevel / typelevel/cats-effect

Laws testing with Arbitrary[IO[A]] from TestInstances

Open
#2,778 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

:beetle: bug
Dominant language
Scala
Stars
2.2k
Forks
576
Avg merge
2d 11h
Merged PRs (30d)
18

Description

I've been recently playing around with migration to CE3. Some of my libraries provide algebras that are law tested. Some of them are meant to be used against IO, thus the laws were also tested against it. In CE2 I would use cats.effect.laws.discipline.arbitrary._ and as discussed on Discord the right replacement would be to have my test suite extend TestInstances and instantiate implicit Ticker().

With this approach my migrated tests wouldn't work out well, but I've learned that Parallel is a tricky one to law test, so I've eliminated that one from my tests.

Despite that effort, I still face issues with law testing quite simple algebras' properties. I've prepared a repository reproducing my issues: https://github.com/majk-p/io-laws-playground/

For the demo I've created two simple algebras, here are the simplified descriptions

trait Storage[F[_], A] {
  def store(a: A): F[Unit]
}

and

trait Consumer[F[_], +A] {
  def consume(f: A => F[Unit]): F[Unit]
}

Say I want to test that the laws of Monoid hold for those two. I've prepared a test suite in:
https://github.com/majk-p/io-laws-playground/blob/master/src/test/scala/net/michalp/StorageLaws.scala

While the tests in DummyStorageLaws (that use IO.pure as ArbitraryIO implementation) and SyncIOStorageLaws (that use TestInstances provided Arbitrary[SyncIO[A]]) or even just OptionLawSpec work perfectly well, implementation based on Arbitrary[IO[A]] from TestInstances fails. The error is more or less like the one below for most tests

[info] IOStorageLaws                                                                                     
[info] monoid laws must hold for Monoid[Storage[IO, Int]]                                  
[error] ! monoid.associative                                                                             
[error]  java.lang.UnsupportedOperationException: Exception raised on property evaluation.> ARG_0: <funct
ion1>                                                                                                    
[error]  > ARG_1: <function1>                                                                            
[error]  > ARG_2: <function1>> Exception: java.lang.UnsupportedOperationException: empty.min             
[error]  The seed is 2ZuVq_ssmSpdX6GM8VKNsiFWtPudTuIZT-WgWQIafPP= (TestContext.scala:98)                 
[error] cats.effect.kernel.testkit.TestContext.nextInterval(TestContext.scala:98)                        
[error] cats.effect.kernel.testkit.TestContext.tickAll(TestContext.scala:179)                            
[error] cats.effect.testkit.TestInstances.unsafeRun(TestInstances.scala:193)                             
[error] cats.effect.testkit.TestInstances.unsafeRun$(TestInstances.scala:183)                            
[error] net.michalp.IOStorageLaws.unsafeRun(StorageLaws.scala:23)                                  
[error] cats.effect.testkit.TestInstances.$anonfun$eqIOA$1(TestInstances.scala:145)
[error] cats.kernel.Eq$$anon$2.eqv(Eq.scala:71)
[error] cats.laws.discipline.eq$.$anonfun$catsLawsEqForFn1Exhaustive$2(Eq.scala:17)
[error] cats.laws.discipline.eq$.$anonfun$catsLawsEqForFn1Exhaustive$2$adapted(Eq.scala:17)
[error] cats.laws.discipline.eq$.$anonfun$catsLawsEqForFn1Exhaustive$1(Eq.scala:17)
[error] cats.laws.discipline.eq$.$anonfun$catsLawsEqForFn1Exhaustive$1$adapted(Eq.scala:17)
[error] cats.kernel.Eq$$anon$5.eqv(Eq.scala:97)
[error] cats.kernel.laws.discipline.package$.catsLawsIsEqToProp(package.scala:11)
[error] cats.kernel.laws.discipline.SemigroupTests.$anonfun$semigroup$2(SemigroupTests.scala:18)
[error] org.scalacheck.Prop$.$anonfun$forAllShrink$2(Prop.scala:768)
[error] org.scalacheck.Prop$.secure(Prop.scala:478)
[error] org.scalacheck.Prop$.result$1(Prop.scala:768)
[error] org.scalacheck.Prop$.$anonfun$forAllShrink$1(Prop.scala:807)
[error] org.scalacheck.Prop$.$anonfun$apply$1(Prop.scala:308)
[error] org.scalacheck.PropFromFun.apply(Prop.scala:21)
[error] org.scalacheck.Prop$.result$1(Prop.scala:769)
[error] org.scalacheck.Prop$.$anonfun$forAllShrink$1(Prop.scala:807)
[error] org.scalacheck.Prop$.$anonfun$apply$1(Prop.scala:308)
[error] org.scalacheck.PropFromFun.apply(Prop.scala:21)
[error] org.scalacheck.Prop$.result$1(Prop.scala:769)
[error] org.scalacheck.Prop$.$anonfun$forAllShrink$1(Prop.scala:807)
[error] org.scalacheck.Prop$.$anonfun$apply$1(Prop.scala:308)
[error] org.scalacheck.PropFromFun.apply(Prop.scala:21)
[error] org.scalacheck.Prop$.$anonfun$delay$1(Prop.scala:483)
[error] org.scalacheck.Prop$.$anonfun$apply$1(Prop.scala:308)
[error] org.scalacheck.PropFromFun.apply(Prop.scala:21)
[error] org.scalacheck.Prop.$anonfun$viewSeed$1(Prop.scala:40)
[error] org.scalacheck.Prop$.$anonfun$apply$1(Prop.scala:308)
[error] org.scalacheck.PropFromFun.apply(Prop.scala:21)
[error] org.scalacheck.Prop$.$anonfun$apply$1(Prop.scala:308)
[error] org.scalacheck.PropFromFun.apply(Prop.scala:21)
[error] org.scalacheck.Test$.workerFun$1(Test.scala:434)
[error] org.scalacheck.Test$.$anonfun$check$2(Test.scala:466)
[error] org.scalacheck.Test$.$anonfun$check$2$adapted(Test.scala:466)
[error] org.scalacheck.Platform$.runWorkers(Platform.scala:40)
[error] org.scalacheck.Test$.check(Test.scala:466)
[error] net.michalp.IOStorageLaws.check(StorageLaws.scala:23)
[error] org.typelevel.discipline.specs2.mutable.Discipline.$anonfun$checkAll$2(Discipline.scala:37)

Please advise if I'm doing something wrong when using library provided arbitraryIO, or is it really a bug?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the failure using the linked io-laws-playground example, starting with src/test/scala/net/michalp/StorageLaws.scala. Trace the stack through cats.effect.kernel.testkit.TestContext.nextInterval and cats.effect.testkit.TestInstances. Done means determining whether Arbitrary[IO[A]] from TestInstances is being used incorrectly or exposes a library bug, with the behavior documented or corrected.

Written by the indexing model from the issue text.

Assessment

Tech stack
scala
Domain
testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.