Lambda facade segment not instanceof Segment
- Dominant language
- JavaScript
- Stars
- 280
- Forks
- 157
- PR merge metrics
- No merged PRs in 30d
Description
I am trying to add subsegments to my Lambda traces for DB queries, but subsegments are not showing up. I initially tried the `AWSXRay.capturePostgres` method, but this proved unsuccessful. I'm seeing trace IDs in the Lambda REPORT logs, but these traces only contain the basic `Invocation` and `Overhead` segments. They do not contain any subsegments I'm attempting to attach.
additional detail collapsed for space
I've iterated quite a bit on things to attempt to get this to work, so the demonstration code below is just where I'm at right now. I've written an XrayManager class which exposes a `wrapQuery` function that accepts a function and returns a new function where a new subsegment is created, the original function is executed, and finally the subsegment is closed. (I realize that this is probably entirely unnecessary, but this is the current state of my code when I discovered the issue, so I want to accurately capture that in the issue)
```js
// my org's logger library, which satisfies the requirements from the README
AWSXRay.setLogger(logger)
class XrayManager {
constructor() {
this.parentSegment = null
}
wrapQuery(queryFn, name, sql) {
return async func => {
this.beginQuery(name, sql)
const res = await queryFn(func)
this.endQuery()
logger.info('completed X-Ray wrapped query', this.parentSegment)
return res
}
}
beginQuery(name, sql) {
this.parentSegment = AWSXRay.getSegment()
if(!this.parentSegment) return
const subsegment = this.parentSegment.addNewSubsegment(name)
AWSXRay.setSegment(subsegment)
subsegment.addSqlData({
sanitized_query: sql,
url: `${HOST}:${PORT}/${DATABASE}`,
user: USER,
})
logger.info('added subsegment', {
subsegment,
root: subsegment.segment,
trace_header: process.env._X_AMZN_TRACE_ID,
})
}
endQuery() {
if(!this.parentSegment) return
const subsegment = AWSXRay.getSegment()
subsegment.close()
AWSXRay.setSegment(subsegment.parent)
}
}
```
The 'added subsegment' log was correctly logging a SubSegment object with the info I expected, so I added the `subsegment.segment` and `subsegment.parent` objects. I confirmed those were both the facade Segment, as expected.
While exploring the `aws-xray-sdk` code try to understand what's happening, I saw that the subsegments are sent in the `Subsegment.flush()` that's called when the Subsegment is closed, and that the emitter [has an `if(this.segment.notTraced !== true)` guard around it](https://github.com/aws/aws-xray-sdk-node/blob/a9d0cf9cbd0328e40f30554f61b4bd5fac08bafc/packages/core/lib/segments/attributes/subsegment.js#L331-L333). I had seen that the `notTraced: true` property was _always_ set on the facade Segment where I was logging it, so I looked to try to understand why that would be. Here I discovered that calling `AWSXRay.getSegment()` should delete the `notTraced` property from the facade Segment if the trace header (from `process.env._X_AMZN_TRACE_ID` has an integer value for Sampled. This led me to tweak my log to include this trace header, and I can see that it has `Sampled=1`, and that the facade Segment still has `notTraced: true`.
I couldn't figure out why this was happening, so I added some `console.log` statements into the library ([compare to](https://github.com/aws/aws-xray-sdk-node/blob/a9d0cf9cbd0328e40f30554f61b4bd5fac08bafc/packages/core/lib/context_utils.js#L96-L110)) and redeployed:
```js
getSegment: function getSegment() {
if (cls_mode) {
var segment = contextUtils.getNamespace(NAMESPACE).get(SEGMENT);
if (!segment) {
contextUtils.contextMissingStrategy.contextMissing('Failed to get the current sub/segment from the context.');
}
else if (segment instanceof Segment && process.env.LAMBDA_TASK_ROOT && segment.facade == true) {
console.log('resolving lambda trace data')
segment.resolveLambdaTraceData();
console.log(`notTraced: ${segment.notTraced}`)
} else {
console.log('segment is present but lambda trace data was not resolved', {
env_var: process.env.LAMBDA_TASK_ROOT,
facade: segment.facade,
segment,
segmentClass: segment.constructor.name,
segmentIsSegment: segment instanceof Segment,
})
}
return segment;
}
else {
contextUtils.contextMissingStrategy.contextMissing('Cannot get sub/segment from context. Not supported in manual mode.');
}
},
```
Logs from this deploy show that, perhaps unexpectedly, while the `segment.facade` property is true and the LAMBDA_TASK_ROOT is set as expected, `segment instanceof Segment` is evaluating as false. This is causing the trace data to not get resolved, which is then causing my lambda's subsegments to get dropped on the floor.
A workaround for this is to immediately call `.resolveLambdaTraceData()` on the facade segment immediately after getting it with `AWSXRay.getSegment()`, but this feels wrong. I would anticipate the `segment instanceof Segment` check within `getSegment` should succeed.
Contributor guide
Assessment
This issue has not been assessed yet.