apple / apple/pkl

Custom ResourceReader implementation in Golang fails to decode read data

Open
#656 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
11.5k
Forks
402
Avg merge
1d 15h
Merged PRs (30d)
20

Description

# context

Hi all, first time submitting a bug report ever. Here's the info:

- What are you trying to achieve?
- Implement a way for pkl to retrieve configuration values from AWS SSM Parameters when `read("ssm:/foo/SomeParameterPath")` is assigned to a pkl resource.
- Working language: `go1.23.0`
- What's the impact of this bug?
- Unmarshaling/resolving values from a module is failing.

## outputs

- `pkl --version: Pkl 0.26.3 (macOS 14.3.1, native)`
- complete error message:
```
java.lang.InterruptedException

–– Pkl Error ––
None (cause has no message)

1 | SomeParameter = read("ssm:ParameterName")
^^^^^^^^^^^^^^^^
at config#SomeParameter (file:///etc/server/config.pkl)

Pkl 0.26.3 (Linux 5.15.0-1057-aws, native)

java.lang.InterruptedException
at java.base@17.0.10/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:386)
at java.base@17.0.10/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2073)
at org.pkl.server.UtilsKt.getUnderlying(Utils.kt:64)
at org.pkl.server.ClientResourceReader.doRead(ClientResourceReader.kt:104)
at org.pkl.server.ClientResourceReader.read(ClientResourceReader.kt:46)
at org.pkl.core.runtime.ResourceManager.lambda$read$1(ResourceManager.java:91)
at java.base@17.0.10/java.util.HashMap.computeIfAbsent(HashMap.java:1220)
at org.pkl.core.runtime.ResourceManager.read(ResourceManager.java:72)
at org.pkl.core.ast.expression.unary.AbstractReadNode.doRead(AbstractReadNode.java:55)
at org.pkl.core.ast.expression.unary.ReadNode.read(ReadNode.java:33)
at org.pkl.core.ast.expression.unary.ReadNodeGen.executeAndSpecialize(ReadNodeGen.java:75)
at org.pkl.core.ast.expression.unary.ReadNodeGen.executeGeneric(ReadNodeGen.java:66)
at org.pkl.core.ast.PklRootNode.executeBody(PklRootNode.java:41)
at org.pkl.core.ast.MemberNode.executeBody(MemberNode.java:53)
at org.pkl.core.ast.member.TypeCheckedPropertyNode.evalTypedObjectCached(TypeCheckedPropertyNode.java:54)
at org.pkl.core.ast.member.TypeCheckedPropertyNodeGen.executeAndSpecialize(TypeCheckedPropertyNodeGen.java:136)
at org.pkl.core.ast.member.TypeCheckedPropertyNodeGen.execute(TypeCheckedPropertyNodeGen.java:101)
at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.executeRootNode(OptimizedCallTarget.java:718)
at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.profiledPERoot(OptimizedCallTarget.java:641)
at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.callBoundary(OptimizedCallTarget.java:574)
at org.graalvm.nativeimage.builder/com.oracle.svm.truffle.api.SubstrateOptimizedCallTarget.invokeCallBoundary(SubstrateOptimizedCallTarget.java:115)
at com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.truffle.SubstrateEnterpriseOptimizedCallTarget.a(stripped:284)
at com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.truffle.SubstrateEnterpriseOptimizedCallTarget.doInvoke(stripped:250)
at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.callIndirect(OptimizedCallTarget.java:486)
at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.call(OptimizedCallTarget.java:467)
at org.graalvm.truffle/com.oracle.truffle.api.nodes.IndirectCallNode$1.call(IndirectCallNode.java:91)
at org.pkl.core.runtime.VmUtils.doReadMember(VmUtils.java:305)
at org.pkl.core.runtime.VmUtils.doReadMember(VmUtils.java:218)
at org.pkl.core.runtime.VmObject.force(VmObject.java:177)
at org.pkl.core.runtime.VmObject.force(VmObject.java:197)
at org.pkl.core.runtime.VmValue.force(VmValue.java:67)
at org.pkl.server.BinaryEvaluator.evaluate$lambda-2(BinaryEvaluator.kt:62)
at org.pkl.core.EvaluatorImpl.lambda$doEvaluate$13(EvaluatorImpl.java:353)
at org.pkl.core.EvaluatorImpl.doEvaluate(EvaluatorImpl.java:301)
at org.pkl.core.EvaluatorImpl.doEvaluate(EvaluatorImpl.java:349)
at org.pkl.server.BinaryEvaluator.evaluate(BinaryEvaluator.kt:58)
at org.pkl.server.Server.handleEvaluate$lambda-0(Server.kt:101)
at java.base@17.0.10/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base@17.0.10/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base@17.0.10/java.lang.Thread.run(Thread.java:842)
at org.graalvm.nativeimage.builder/com.oracle.svm.core.thread.PlatformThreads.threadStartRoutine(PlatformThreads.java:807)
at org.graalvm.nativeimage.builder/com.oracle.svm.core.posix.thread.PosixPlatformThreads.pthreadStartRoutine(PosixPlatformThreads.java:210)

goroutine 1 [running]:
cdr-getter-api/configuration.AppConfig()
/app/configuration/configuration.go:51 +0x1f6
cdr-getter-api/controllers.init()
/app/controllers/globals.go:24 +0x17
bash-5.2# An unexpected error has occurred. Would you mind filing a bug report?

java.lang.NullPointerException
at org.pkl.server.ClientResourceReader$doRead$1$1$1.invoke(ClientResourceReader.kt:94)
at org.pkl.server.ClientResourceReader$doRead$1$1$1.invoke(ClientResourceReader.kt:88)
at org.pkl.server.MessageTransports$AbstractMessageTransport.accept(MessageTransports.kt:98)
at org.pkl.server.MessageTransports$EncodingMessageTransport.doStart(MessageTransports.kt:50)
at org.pkl.server.MessageTransports$AbstractMessageTransport.start(MessageTransports.kt:110)
at org.pkl.server.Server.start(Server.kt:42)
at org.pkl.cli.CliServer.doRun(CliServer.kt:29)
at org.pkl.commons.cli.CliCommand.run(CliCommand.kt:44)
at org.pkl.cli.commands.ServerCommand.run(ServerCommand.kt:30)
at com.github.ajalt.clikt.parsers.Parser.parse(Parser.kt:198)
at com.github.ajalt.clikt.parsers.Parser.parse(Parser.kt:211)
at com.github.ajalt.clikt.parsers.Parser.parse(Parser.kt:18)
at com.github.ajalt.clikt.core.CliktCommand.parse(CliktCommand.kt:400)
at com.github.ajalt.clikt.core.CliktCommand.parse$default(CliktCommand.kt:397)
at com.github.ajalt.clikt.core.CliktCommand.main(CliktCommand.kt:415)
at com.github.ajalt.clikt.core.CliktCommand.main(CliktCommand.kt:440)
at org.pkl.cli.Main$main$1.invoke(Main.kt:39)
at org.pkl.cli.Main$main$1.invoke(Main.kt:27)
at org.pkl.commons.cli.CliMainKt.cliMain(CliMain.kt:33)
at org.pkl.cli.Main.main(Main.kt:27)
```
- Reproducing:
- contents of `.pkl` file:
```
SomeParameterValue = read("ssm:/foo/SomeParameterPath")
```
- implement a a custom evaluator whose `Read` method returns a byte array as shown below:
```golang
// from the pkl.ResourceReader requirement of pkl.Reader interface
func (r PklResourceReader) Read(url url.URL) ([]byte, error) {
// v is type *string
v, err := GetDecryptedParameter(context.Background(), url.Path)
if err != nil {
return nil, err
}
if v == nil {
return make([]byte, 0), nil
}
var b []byte
// if err := msgpack.NewEncoder(bytes.NewBuffer(b)).EncodeString(*v); err != nil {
// return nil, err
// }
return b, nil
}
```
- load the file using the pkl module's `.Load()` method.
- note that the contents of `b` do make a difference:
- on an empty array, the error above is shown.
- on a non-empty array like `[]byte(*v)`, `[]byte("foo")`, the following error returns:
```
msgpack: invalid code=94 decoding string/bytes length
```

Any pointers/insight on this would be appreciated. Thank you.

Contributor guide

Open the contributing guide

Research direction

Reproduce the failure with the custom Pkl ResourceReader.Read method and the module .Load() call. Then trace the resource response through ClientResourceReader.kt and the pkl.Reader requirement, comparing empty, raw, and encoded byte arrays. Done means the AWS SSM-backed resource decodes successfully without the InterruptedException, NullPointerException, or MsgPack decoding error.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, java, kotlin
Domain
api, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.