apache / apache/dubbo

Triple attachment does not support non-ASCII characters

Open
#12,904 3 comments 0 reactions 1 assignee Claimed by @oxsean View on GitHub
component/sdk help wanted type/enhancement
Dominant language
Java
Stars
41.6k
Forks
26.4k
Avg merge
15h 13m
Merged PRs (30d)
4

Description

- [x] I have searched the [issues](https://github.com/apache/dubbo/issues) of this repository and believe that this is not a duplicate.

### Environment

* Dubbo version: 3.2.5
* Java version: 1.8

### Steps to reproduce this issue

1. Insert a value containing non-ASCII characters in the attachment
2. Get the attachment in the provider, where non-ascii characters are converted to "?"

reproduce this issue.
```
import org.apache.dubbo.rpc.Filter;
import org.apache.dubbo.rpc.Invocation;
import org.apache.dubbo.rpc.Invoker;
import org.apache.dubbo.rpc.Result;
import org.apache.dubbo.rpc.RpcContext;
import org.apache.dubbo.rpc.RpcException;

public class ClientContextAppender implements Filter {
public ClientContextAppender() {
}

public Result invoke(Invoker invoker, Invocation invocation) throws RpcException {
RpcContext context = RpcContext.getClientAttachment();
context.setAttachment("non-Ascii", "zh这是中文zh");
return invoker.invoke(invocation);
}
}
```
image

### Expected Behavior

Consistent with the attachment of the dubbo protocol, The triple protocol can also pass values containing non-ASCII characters

### Actual Behavior

Non-ASCII characters are converted to '?'

### Problem Code
The netty's huffman encoding condition
image
#### Not hit netty's huffman encoding condition

When netty does not hit the huffman encoding of the header, it will convert the non-ascii code to '?'

```
c2b:1857, AsciiString (io.netty.util)
writeAscii:1188, ByteBufUtil (io.netty.buffer)
setCharSequence0:717, AbstractByteBuf (io.netty.buffer)
writeCharSequence:1187, AbstractByteBuf (io.netty.buffer)
encodeStringLiteral:279, HpackEncoder (io.netty.handler.codec.http2)
encodeLiteral:306, HpackEncoder (io.netty.handler.codec.http2)
encodeHeader:194, HpackEncoder (io.netty.handler.codec.http2)
encodeHeadersIgnoreMaxHeaderListSize:145, HpackEncoder (io.netty.handler.codec.http2)
encodeHeadersEnforceMaxHeaderListSize:137, HpackEncoder (io.netty.handler.codec.http2)
encodeHeaders:118, HpackEncoder (io.netty.handler.codec.http2)
encodeHeaders:74, DefaultHttp2HeadersEncoder (io.netty.handler.codec.http2)
...
```
image

#### Hit netty's huffman encoding condition
4.1.86.Final Netty has a [bug](https://github.com/netty/netty/issues/13540) in the conversion

When netty hits the huffman encoding condition of the header, non-ASCII characters are cleaned to the ASCII range
image
Such a convert may cause unexpected errors
Non-ASCII characters may be converted to characters less than 32
For example, envoy will check the value of the header, and if it finds characters less than 32, it will report an error

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.