apache / apache/brpc

异步情况下的span传递问题

Open
#1,010 10 comments 0 reactions 0 assignees View on GitHub
feature
Dominant language
C++
Stars
17.6k
Forks
4.1k
Avg merge
2d 12h
Merged PRs (30d)
69

Description

**Describe the bug (描述bug)**
关于异步情况下 clientSpan的cr时间问题,对应span中的字段为:_received_real_us
观察到这个字段是在收包阶段设置。若brpc作为proxy,接收上游请求,之后发起异步调用,这时Server Controller会被销毁,由此调用Span的submit方法,此种情况下,异步调用的clientSpan会跟随ServerSpan一同销毁,由此会出现cr=0的情况。
Span销毁后,只是调用ReturnToObject丢回对象池,异步回调结束后,仍然可以拿到ParentSpan的指针,同时Client Controller事实上仍然是持有 client Span的指针的,这里设计是否存在问题?

**To Reproduce (复现方法)**
异步调用,一段时间后返回,cr时间为0.

**Expected behavior (期望行为)**
clientSpan的销毁时机,应该在异步调用返回之后。

**Versions (各种版本)**
OS:
Compiler:
brpc:
protobuf:

**Additional context/screenshots (更多上下文/截图)**

V10 0104 10:54:57.397914 6221 18446744073709551615 src/brpc/span.cpp:316 [destroy] 0x4204240
V10 0104 10:54:57.397921 6221 18446744073709551615 src/brpc/span.cpp:323 [destroy] 0x4204000
V10 0104 10:54:58.394481 5744 4294976000 src/brpc/controller.cpp:1244 [SubmitSpan] this span:0x4204240
V10 0104 10:54:58.394501 5744 4294976000 src/brpc/controller.cpp:1245 [SubmitSpan] parent:0x4204000

这是打印出的日志信息(行号不一致请忽略,有别处的改动,但未影响span),异步情况下, proxy的server span调用destroy,同时destroy了所有的client span,异步调用返回时,又再次打印出这两个已经销毁的span(parent与client都已给出),可以确认是存在问题的。
这是代码位置:
![image](https://user-images.githubusercontent.com/4979167/71759803-a941a580-2eed-11ea-89c5-bad271693990.png)
与日志结合看,这里在span销毁后再次操作了span对象,因为returnToObject机制未销毁该指针,所以能够访问,事实上这样的操作应该是在操作野指针。

Contributor guide

Open the contributing guide

Research direction

Start with src/brpc/span.cpp and src/brpc/controller.cpp, especially the destroy, ReturnToObject, and SubmitSpan paths named in the report. Reproduce a delayed asynchronous call and compare the logged span addresses and _received_real_us handling. Done means the client span remains valid until the asynchronous callback returns and the resulting client span records a nonzero receive time.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend-api-design, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.