typelevel / typelevel/weaver-test

UTF8 string too large when comparing 2 big objects

Open
#79 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Scala
Stars
85
Forks
15
PR merge metrics
No merged PRs in 30d

Description

This issue was copied over from: https://github.com/disneystreaming/weaver-test/issues/760
It was opened by: PaNaVTEC


The following attached test will crash when executing, while this is a toy example it can happen if you try to compare equality on two massive objects, I managed to replicate it in a real world™ scenario by trying to compare 2 stripe invoices. The crash goes away if the long inlined object (string in the example below) is extracted to a local variable and you do:

// Working example
val expectedStr = "... my long str ... "
"" == expectedStr

Affected example: https://pastebin.com/0d48ACec

Trace:

java.lang.IllegalArgumentException: UTF8 string too large
	at scala.tools.asm.ByteVector.putUTF8(ByteVector.java:255)
	at scala.tools.asm.SymbolTable.addConstantUtf8(SymbolTable.java:774)
	at scala.tools.asm.SymbolTable.addConstantUtf8Reference(SymbolTable.java:1007)
	at scala.tools.asm.SymbolTable.addConstantString(SymbolTable.java:604)
	at scala.tools.asm.SymbolTable.addConstant(SymbolTable.java:474)
	at scala.tools.asm.MethodWriter.visitLdcInsn(MethodWriter.java:1280)
	at scala.tools.asm.tree.LdcInsnNode.accept(LdcInsnNode.java:75)
	at scala.tools.asm.tree.InsnList.accept(InsnList.java:144)
	at scala.tools.asm.tree.MethodNode.accept(MethodNode.java:749)
	at scala.tools.asm.tree.MethodNode.accept(MethodNode.java:647)
	at scala.tools.asm.tree.ClassNode.accept(ClassNode.java:468)
	at dotty.tools.backend.jvm.PostProcessor.serializeClass(PostProcessor.scala:71)
	at dotty.tools.backend.jvm.PostProcessor.postProcessAndSendToDisk$$anonfun$2(PostProcessor.scala:31)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
	at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:576)
	at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:574)
	at scala.collection.AbstractIterable.foreach(Iterable.scala:933)
	at scala.collection.IterableOps$WithFilter.foreach(Iterable.scala:903)
	at dotty.tools.backend.jvm.PostProcessor.postProcessAndSendToDisk(PostProcessor.scala:48)
	at dotty.tools.backend.jvm.GenBCode.run(GenBCode.scala:80)
	at dotty.tools.dotc.core.Phases$Phase.runOn$$anonfun$1(Phases.scala:327)
	at scala.collection.immutable.List.map(List.scala:246)
	at dotty.tools.dotc.core.Phases$Phase.runOn(Phases.scala:331)
	at dotty.tools.backend.jvm.GenBCode.runOn(GenBCode.scala:87)
	at dotty.tools.dotc.Run.runPhases$1$$anonfun$1(Run.scala:246)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
	at scala.collection.ArrayOps$.foreach$extension(ArrayOps.scala:1323)
	at dotty.tools.dotc.Run.runPhases$1(Run.scala:262)
	at dotty.tools.dotc.Run.compileUnits$$anonfun$1(Run.scala:270)
	at dotty.tools.dotc.Run.compileUnits$$anonfun$adapted$1(Run.scala:279)
	at dotty.tools.dotc.util.Stats$.maybeMonitored(Stats.scala:71)
	at dotty.tools.dotc.Run.compileUnits(Run.scala:279)
	at dotty.tools.dotc.Run.compileSources(Run.scala:194)
	at dotty.tools.dotc.Run.compile(Run.scala:179)
	at dotty.tools.dotc.Driver.doCompile(Driver.scala:37)
	at dotty.tools.xsbt.CompilerBridgeDriver.run(CompilerBridgeDriver.java:136)
	at dotty.tools.xsbt.CompilerBridge.run(CompilerBridge.java:22)
	at sbt.internal.inc.AnalyzingCompiler.compile(AnalyzingCompiler.scala:91)
	at sbt.internal.inc.MixedAnalyzingCompiler.$anonfun$compile$7(MixedAnalyzingCompiler.scala:193)
	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
	at sbt.internal.inc.MixedAnalyzingCompiler.timed(MixedAnalyzingCompiler.scala:248)
	at sbt.internal.inc.MixedAnalyzingCompiler.$anonfun$compile$4(MixedAnalyzingCompiler.scala:183)
	at sbt.internal.inc.MixedAnalyzingCompiler.$anonfun$compile$4$adapted(MixedAnalyzingCompiler.scala:163)
	at sbt.internal.inc.JarUtils$.withPreviousJar(JarUtils.scala:239)
	at sbt.internal.inc.MixedAnalyzingCompiler.compileScala$1(MixedAnalyzingCompiler.scala:163)
	at sbt.internal.inc.MixedAnalyzingCompiler.compile(MixedAnalyzingCompiler.scala:211)
	at sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileInternal$1(IncrementalCompilerImpl.scala:534)
	at sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileInternal$1$adapted(IncrementalCompilerImpl.scala:534)
	at sbt.internal.inc.Incremental$.$anonfun$apply$5(Incremental.scala:180)
	at sbt.internal.inc.Incremental$.$anonfun$apply$5$adapted(Incremental.scala:178)
	at sbt.internal.inc.Incremental$$anon$2.run(Incremental.scala:464)
	at sbt.internal.inc.IncrementalCommon$CycleState.next(IncrementalCommon.scala:116)
	at sbt.internal.inc.IncrementalCommon$$anon$1.next(IncrementalCommon.scala:56)
	at sbt.internal.inc.IncrementalCommon$$anon$1.next(IncrementalCommon.scala:52)
	at sbt.internal.inc.IncrementalCommon.cycle(IncrementalCommon.scala:263)
	at sbt.internal.inc.Incremental$.$anonfun$incrementalCompile$8(Incremental.scala:419)
	at sbt.internal.inc.Incremental$.withClassfileManager(Incremental.scala:506)
	at sbt.internal.inc.Incremental$.incrementalCompile(Incremental.scala:406)
	at sbt.internal.inc.Incremental$.apply(Incremental.scala:172)
	at sbt.internal.inc.IncrementalCompilerImpl.compileInternal(IncrementalCompilerImpl.scala:534)
	at sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileIncrementally$1(IncrementalCompilerImpl.scala:488)
	at sbt.internal.inc.IncrementalCompilerImpl.handleCompilationError(IncrementalCompilerImpl.scala:332)
	at sbt.internal.inc.IncrementalCompilerImpl.compileIncrementally(IncrementalCompilerImpl.scala:425)
	at sbt.internal.inc.IncrementalCompilerImpl.compile(IncrementalCompilerImpl.scala:137)
	at sbt.Defaults$.compileIncrementalTaskImpl(Defaults.scala:2371)
	at sbt.Defaults$.$anonfun$compileIncrementalTask$2(Defaults.scala:2321)
	at sbt.internal.server.BspCompileTask$.$anonfun$compute$1(BspCompileTask.scala:31)
	at sbt.internal.io.Retry$.apply(Retry.scala:47)
	at sbt.internal.io.Retry$.apply(Retry.scala:29)
	at sbt.internal.io.Retry$.apply(Retry.scala:24)
	at sbt.internal.server.BspCompileTask$.compute(BspCompileTask.scala:31)
	at sbt.Defaults$.$anonfun$compileIncrementalTask$1(Defaults.scala:2319)
	at scala.Function1.$anonfun$compose$1(Function1.scala:49)
	at sbt.internal.util.$tilde$greater.$anonfun$$u2219$1(TypeFunctions.scala:63)
	at sbt.std.Transform$$anon$4.work(Transform.scala:69)
	at sbt.Execute.$anonfun$submit$2(Execute.scala:283)
	at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:24)
	at sbt.Execute.work(Execute.scala:292)
	at sbt.Execute.$anonfun$submit$1(Execute.scala:283)
	at sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:265)
	at sbt.CompletionService$$anon$2.call(CompletionService.scala:65)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:840)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the affected test from the linked Pastebin example, comparing the inline large string with the local-variable workaround. Trace the compilation failure through the dotty JVM backend and the scala.tools.asm UTF8 error; done means the inline comparison compiles without the crash while equality behavior remains correct.

Written by the indexing model from the issue text.

Assessment

Tech stack
scala
Domain
compilers, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.