apache / apache/dubbo-go

[PROPOSAL] Explore Apache Fory as a pluggable serialization codec for Triple non-IDL calls

Open
#3,497 0 comments 0 reactions 0 assignees View on GitHub
✏️ Feature 4.0.0 proposal
Dominant language
Go
Stars
5k
Forks
1k
Avg merge
2d 8h
Merged PRs (30d)
31

Description

## ✨ Proposal Summary

Explore introducing [Apache Fory](https://fory.apache.org/) as a new pluggable serialization codec for Triple non-IDL calls.

Triple currently supports Hessian2 and MsgPack as inner codecs for non-IDL arguments carried by the existing Protobuf `TripleRequestWrapper`. Hessian2 provides Dubbo compatibility but may become a serialization performance bottleneck. MsgPack is lightweight, but its cross-language object model and schema-evolution capabilities are less aligned with Hessian2-style generic object serialization.

Apache Fory is a candidate because it provides:

- Cross-language serialization between Go and Java.
- Direct serialization of application objects without requiring Protobuf IDL.
- Type registration and schema-evolution support.
- Polymorphism, shared references, and circular-reference support.
- Performance-oriented Go and Java implementations.

This proposal is exploratory. It does not propose removing Hessian2 or changing the default serialization before compatibility and performance are validated.

## Serialization layers and scope

Fory and the final Protobuf wrapper belong to two different serialization layers.

For Triple non-IDL calls, the data path is:

```text
Application argument objects

│ Hessian2 / MsgPack / future Fory

Serialized argument bytes ([][]byte)

│ stored in TripleRequestWrapper.Args

TripleRequestWrapper

│ Protobuf encoding

Triple wire message
```

Fory would replace only the inner argument serialization currently handled by Hessian2 or MsgPack. It is responsible for converting concrete business values, such as a Go struct or Java POJO, into the byte slices stored in the wrapper.

The outer `TripleRequestWrapper` remains encoded with Protobuf. It acts as a stable envelope carrying fields such as the inner serialization type, argument type names, and serialized argument bytes. The outer Protobuf codec does not interpret the Fory/Hessian2/MsgPack payload itself.

The IDL and non-IDL paths should therefore be distinguished explicitly:

```text
IDL:
Business Protobuf message
→ Protobuf
→ Triple

non-IDL:
Business objects
→ Fory / Hessian2 / MsgPack
→ TripleRequestWrapper
→ Protobuf
→ Triple
```

This proposal only explores replacing or extending the inner codec in the non-IDL path. It does not propose replacing the outer Protobuf wrapper or changing the normal Protobuf IDL path.

## 🛠️ Implementation Approach

### 1. Add Fory as an optional inner codec

Introduce a new serialization name:

```text
serialization=fory
```

Keep the Triple outer wire format unchanged:

```text
TripleRequestWrapper (Protobuf)
└── Args (Fory xlang payloads)
```

The wrapper should carry `SerializeType = "fory"`, and the server should resolve the inner codec deterministically from that value. This should align with #3496.

Hessian2 must remain available for compatibility with existing Dubbo clients and providers.

### 2. Use Fory cross-language mode

Triple interoperability requires Fory xlang mode rather than Go-native mode. The PoC should evaluate:

- xlang + compatible mode.
- xlang + non-compatible mode, only when both sides use an identical schema.

Codec instances should be reused instead of creating a new Fory instance per request.

### 3. Define type mapping and registration

Validate and document the Go ↔ Java contract for:

- Primitive values and strings.
- Structs/POJOs.
- Slices, arrays, maps, and nested collections.
- Pointers and null values.
- Interface/polymorphic values.
- Shared and circular references.
- Time-related types.
- Exceptions and generic invocation results.
- Type registration by stable name or ID.
- Schema evolution across client/provider versions.

Unknown or unregistered types must return explicit serialization errors rather than silently falling back to another codec.

### 4. Build an interoperability test matrix

Add Go ↔ Java tests covering:

- Go consumer → Java provider.
- Java consumer → Go provider.
- Unary non-IDL calls.
- Generic invocation request and response values.
- Compatible schema evolution: added, removed, and reordered fields.
- Unsupported, unknown, corrupted, and mismatched payloads.
- Coexistence of Hessian2, MsgPack, and Fory clients during migration.

### 5. Benchmark the real Triple path

Compare at least:

- Hessian2.
- MsgPack.
- Fory xlang compatible.
- Fory xlang non-compatible.
- Protobuf IDL as a reference baseline.

Measure encode/decode latency, end-to-end Triple unary latency, throughput, allocations, allocated bytes, and payload size across small, medium, nested, collection-heavy, and polymorphic payloads.

Go-native Fory results should not be used to justify the cross-language Triple path.

### 6. Security and resource limits

Review safe decoding behavior for network-provided payloads, including:

- Maximum nesting depth.
- Maximum type/schema metadata size.
- Maximum collection and payload size.
- Allowed type registry.
- Rejection of unknown or disabled serialization types.
- Avoiding arbitrary type construction from untrusted input.

## 📚 Additional Context

Related issue:

- #3496: dispatch Triple non-IDL wrapper decoding by `SerializeType`.

Suggested delivery stages:

1. A standalone Go/Java interoperability and benchmark PoC.
2. An experimental Fory codec behind explicit configuration.
3. Documentation and samples.
4. A separate decision, based on measured results, about production status or default selection.

### Acceptance criteria for the exploration

- [ ] A reusable Fory xlang codec prototype exists for Triple non-IDL requests and responses.
- [ ] Go ↔ Java interoperability tests cover the supported type matrix.
- [ ] Benchmarks compare Fory with Hessian2 and MsgPack on representative payloads.
- [ ] Schema evolution, type registration, concurrency, and security constraints are documented.
- [ ] Fory can coexist with existing codecs and is selected deterministically through `SerializeType`.
- [ ] The exploration concludes with a documented go/no-go recommendation and known compatibility gaps.

Contributor guide

Open the contributing guide

Research direction

Start with related issue #3496 and map the existing Triple non-IDL codec dispatch and TripleRequestWrapper path. Build the proposed standalone Go/Java interoperability and benchmark PoC first, covering the listed type matrix, codec coexistence, schema evolution, and security limits. Done means documented compatibility results, benchmark comparisons, known gaps, and a go/no-go recommendation.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, java
Domain
backend-api-design, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.