awslabs / awslabs/aws-embedded-metrics-node

Allow for a "softFail" configuration option

Open
#170 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
263
Forks
39
PR merge metrics
No merged PRs in 30d

Description

This is a feature request for an additional configuration parameter, "softFail". When true, `AgentSink` would suppress any `ECONNREFUSED` error occurring when sending a message to the CloudWatch agent and just log it instead.

Here's our rationale for this change: We have code that runs in both Lambda environments (where we always want metrics to be logged), but also in local environments for diagnostics and exploration purposes (where we don't care if metrics are logged). In other words, some of our metric logging is "best effort", not "mission critical".

We know we can configure the `environmentOverride` to be `Local`, which will log the metrics to the console and not fail. But those messages are distracting, and doing so via an environment variable is awkward. We can also wrap the library calls to add conditionals to only publish metrics when we detect the right environment, or even just suppress the thrown exception ourselves. But this "best effort" scenario seems like a common enough use case that a configuration parameter could be helpful.

We have a working fork with a rough draft of the changes that suit our needs. Is there interest in including a feature like this in a `v4.3+` release? If so, I'd be happy to spin up a pull request.

Contributor guide

Open the contributing guide

Research direction

Start by locating AgentSink and its handling of ECONNREFUSED, then trace how configuration parameters are defined and passed through the library. Add the softFail behavior described in the issue and verify that connection-refused errors are logged rather than thrown when it is enabled, while normal failures remain unchanged.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
observability-sre
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.