[test] Tests for custom sections

I am the champion of the Branch Hinting proposal, and I am thinking about the requirement of adding tests for entering phase 3.

There is currently no infrastructure in the reference interpreter (or any other Wasm engine I think) to validate the parsing of custom section payloads. Not even for the names section (I was hoping I could take inspiration from how that is handled, but all the tests for the name section are about the names themselves, not the binary format).

The Instrument Tracing proposal is also going to use a custom section, so this discussion should be relevant for it too.

I am not familiar with OCaml, nor the interpreter codebase specifically, but I think that a solution could be to add new "commands" to the extended wast syntax used in the tests:

• assert_custom_valid <custom section content> : assert given well-known custom section parses successfully
• assert_custom_invalid <custom section content> : assert given well-known custom section parses unsuccessfully

This would allow to add tests for the names section binary format, in addition to the Branch Hinting and Instrument Tracing proposals (of course for each of these, we will need to write the validation logic in the interpreter).

Before trying to experiment with this idea I would like to hear what people think about it, and if there are alternatives.

As added context, here is an extract from the CG-06-08 meeting where we briefly talked about this:

DS: in general sense, there’s no good way for end-to-end test, non-observable behavior in the engine. One way in the standard unit testing system is to design a system with some other kind of side entry api that will let you observe these differences. In this case that could mean providing some way in the reference interpreter to observe some of these expected effects, not sure if that’s something we want.

YI: right now, for names section, it’s kind of similar, how is that handled in the test.

DS: having no test is another answer

DP (chat): Well, you could run all tests with and without hinting to ensure (1) same behavior and (2) branch hinting isn't slower

KM: is there a flag to tell the interpret to treat custom section failures as failures? Can give you an idea of how the custom section is parsed, you won’t test the actual feature of emitting the hints. I can see parsing tests, not sure about code-gen tests. Engines probably have a way to check these, e.g. assembler. Not really portable.

YI: will open issue, and see if there is easy way to add parsing test

The issue names no specific files or existing tests. Start by examining the reference interpreter's extended wast test syntax and custom-section parsing, then compare how the names section is handled. Done would require an agreed approach and tests that distinguish valid from invalid custom-section payloads.