Custom ResourceReader implementation in Golang fails to decode read data
- Dominant language
- Java
- Stars
- 11.5k
- Forks
- 402
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 20
Description
# context
Hi all, first time submitting a bug report ever. Here's the info:
- What are you trying to achieve?
- Implement a way for pkl to retrieve configuration values from AWS SSM Parameters when `read("ssm:/foo/SomeParameterPath")` is assigned to a pkl resource.
- Working language: `go1.23.0`
- What's the impact of this bug?
- Unmarshaling/resolving values from a module is failing.
## outputs
- `pkl --version: Pkl 0.26.3 (macOS 14.3.1, native)`
- complete error message:
```
java.lang.InterruptedException
–– Pkl Error ––
None (cause has no message)
1 | SomeParameter = read("ssm:ParameterName")
^^^^^^^^^^^^^^^^
at config#SomeParameter (file:///etc/server/config.pkl)
Pkl 0.26.3 (Linux 5.15.0-1057-aws, native)
java.lang.InterruptedException
at java.base@17.0.10/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:386)
at java.base@17.0.10/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2073)
at org.pkl.server.UtilsKt.getUnderlying(Utils.kt:64)
at org.pkl.server.ClientResourceReader.doRead(ClientResourceReader.kt:104)
at org.pkl.server.ClientResourceReader.read(ClientResourceReader.kt:46)
at org.pkl.core.runtime.ResourceManager.lambda$read$1(ResourceManager.java:91)
at java.base@17.0.10/java.util.HashMap.computeIfAbsent(HashMap.java:1220)
at org.pkl.core.runtime.ResourceManager.read(ResourceManager.java:72)
at org.pkl.core.ast.expression.unary.AbstractReadNode.doRead(AbstractReadNode.java:55)
at org.pkl.core.ast.expression.unary.ReadNode.read(ReadNode.java:33)
at org.pkl.core.ast.expression.unary.ReadNodeGen.executeAndSpecialize(ReadNodeGen.java:75)
at org.pkl.core.ast.expression.unary.ReadNodeGen.executeGeneric(ReadNodeGen.java:66)
at org.pkl.core.ast.PklRootNode.executeBody(PklRootNode.java:41)
at org.pkl.core.ast.MemberNode.executeBody(MemberNode.java:53)
at org.pkl.core.ast.member.TypeCheckedPropertyNode.evalTypedObjectCached(TypeCheckedPropertyNode.java:54)
at org.pkl.core.ast.member.TypeCheckedPropertyNodeGen.executeAndSpecialize(TypeCheckedPropertyNodeGen.java:136)
at org.pkl.core.ast.member.TypeCheckedPropertyNodeGen.execute(TypeCheckedPropertyNodeGen.java:101)
at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.executeRootNode(OptimizedCallTarget.java:718)
at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.profiledPERoot(OptimizedCallTarget.java:641)
at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.callBoundary(OptimizedCallTarget.java:574)
at org.graalvm.nativeimage.builder/com.oracle.svm.truffle.api.SubstrateOptimizedCallTarget.invokeCallBoundary(SubstrateOptimizedCallTarget.java:115)
at com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.truffle.SubstrateEnterpriseOptimizedCallTarget.a(stripped:284)
at com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.truffle.SubstrateEnterpriseOptimizedCallTarget.doInvoke(stripped:250)
at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.callIndirect(OptimizedCallTarget.java:486)
at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.call(OptimizedCallTarget.java:467)
at org.graalvm.truffle/com.oracle.truffle.api.nodes.IndirectCallNode$1.call(IndirectCallNode.java:91)
at org.pkl.core.runtime.VmUtils.doReadMember(VmUtils.java:305)
at org.pkl.core.runtime.VmUtils.doReadMember(VmUtils.java:218)
at org.pkl.core.runtime.VmObject.force(VmObject.java:177)
at org.pkl.core.runtime.VmObject.force(VmObject.java:197)
at org.pkl.core.runtime.VmValue.force(VmValue.java:67)
at org.pkl.server.BinaryEvaluator.evaluate$lambda-2(BinaryEvaluator.kt:62)
at org.pkl.core.EvaluatorImpl.lambda$doEvaluate$13(EvaluatorImpl.java:353)
at org.pkl.core.EvaluatorImpl.doEvaluate(EvaluatorImpl.java:301)
at org.pkl.core.EvaluatorImpl.doEvaluate(EvaluatorImpl.java:349)
at org.pkl.server.BinaryEvaluator.evaluate(BinaryEvaluator.kt:58)
at org.pkl.server.Server.handleEvaluate$lambda-0(Server.kt:101)
at java.base@17.0.10/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base@17.0.10/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base@17.0.10/java.lang.Thread.run(Thread.java:842)
at org.graalvm.nativeimage.builder/com.oracle.svm.core.thread.PlatformThreads.threadStartRoutine(PlatformThreads.java:807)
at org.graalvm.nativeimage.builder/com.oracle.svm.core.posix.thread.PosixPlatformThreads.pthreadStartRoutine(PosixPlatformThreads.java:210)
goroutine 1 [running]:
cdr-getter-api/configuration.AppConfig()
/app/configuration/configuration.go:51 +0x1f6
cdr-getter-api/controllers.init()
/app/controllers/globals.go:24 +0x17
bash-5.2# An unexpected error has occurred. Would you mind filing a bug report?
java.lang.NullPointerException
at org.pkl.server.ClientResourceReader$doRead$1$1$1.invoke(ClientResourceReader.kt:94)
at org.pkl.server.ClientResourceReader$doRead$1$1$1.invoke(ClientResourceReader.kt:88)
at org.pkl.server.MessageTransports$AbstractMessageTransport.accept(MessageTransports.kt:98)
at org.pkl.server.MessageTransports$EncodingMessageTransport.doStart(MessageTransports.kt:50)
at org.pkl.server.MessageTransports$AbstractMessageTransport.start(MessageTransports.kt:110)
at org.pkl.server.Server.start(Server.kt:42)
at org.pkl.cli.CliServer.doRun(CliServer.kt:29)
at org.pkl.commons.cli.CliCommand.run(CliCommand.kt:44)
at org.pkl.cli.commands.ServerCommand.run(ServerCommand.kt:30)
at com.github.ajalt.clikt.parsers.Parser.parse(Parser.kt:198)
at com.github.ajalt.clikt.parsers.Parser.parse(Parser.kt:211)
at com.github.ajalt.clikt.parsers.Parser.parse(Parser.kt:18)
at com.github.ajalt.clikt.core.CliktCommand.parse(CliktCommand.kt:400)
at com.github.ajalt.clikt.core.CliktCommand.parse$default(CliktCommand.kt:397)
at com.github.ajalt.clikt.core.CliktCommand.main(CliktCommand.kt:415)
at com.github.ajalt.clikt.core.CliktCommand.main(CliktCommand.kt:440)
at org.pkl.cli.Main$main$1.invoke(Main.kt:39)
at org.pkl.cli.Main$main$1.invoke(Main.kt:27)
at org.pkl.commons.cli.CliMainKt.cliMain(CliMain.kt:33)
at org.pkl.cli.Main.main(Main.kt:27)
```
- Reproducing:
- contents of `.pkl` file:
```
SomeParameterValue = read("ssm:/foo/SomeParameterPath")
```
- implement a a custom evaluator whose `Read` method returns a byte array as shown below:
```golang
// from the pkl.ResourceReader requirement of pkl.Reader interface
func (r PklResourceReader) Read(url url.URL) ([]byte, error) {
// v is type *string
v, err := GetDecryptedParameter(context.Background(), url.Path)
if err != nil {
return nil, err
}
if v == nil {
return make([]byte, 0), nil
}
var b []byte
// if err := msgpack.NewEncoder(bytes.NewBuffer(b)).EncodeString(*v); err != nil {
// return nil, err
// }
return b, nil
}
```
- load the file using the pkl module's `.Load()` method.
- note that the contents of `b` do make a difference:
- on an empty array, the error above is shown.
- on a non-empty array like `[]byte(*v)`, `[]byte("foo")`, the following error returns:
```
msgpack: invalid code=94 decoding string/bytes length
```
Any pointers/insight on this would be appreciated. Thank you.
Contributor guide
Research direction
Reproduce the failure with the custom Pkl ResourceReader.Read method and the module .Load() call. Then trace the resource response through ClientResourceReader.kt and the pkl.Reader requirement, comparing empty, raw, and encoded byte arrays. Done means the AWS SSM-backed resource decodes successfully without the InterruptedException, NullPointerException, or MsgPack decoding error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, java, kotlin
- Domain
- api, backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100