optimizely / optimizely/agent

[BUG] unable to fetch fresh datafile (consider rechecking SDK key), status code: 403 Forbidden during init

Open
#424 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Go
Stars
34
Forks
26
PR merge metrics
No merged PRs in 30d

Description

Is there an existing issue for this?
  • I have searched the existing issues
Agent Version

4.1.0 container running on ECS/Fargate

Current Behavior

We kept seeing these in our logs:

{
    "level": "error",
    "instance": "Instance-2",
    "name": "PollingProjectConfigManager",
    "sdkKey": "undefined",
    "time": "2024-09-15T15:25:48Z",
    "message": "unable to fetch fresh datafile: 403 Forbidden"
}
{
    "level": "error",
    "requestId": "cf4cbbb4-126d-4efb-816b-4b1932c13540",
    "sdkKey": "undefined",
    "error": "unable to fetch fresh datafile (consider rechecking SDK key), status code: 403 Forbidden",
    "time": "2024-09-15T15:25:48Z",
    "message": "Initializing OptimizelyClient"
}
{
    "level": "info",
    "instance": "Instance-1",
    "name": "OptimizelyClient",
    "sdkKey": "<sdk-key>",
    "time": "2024-09-15T15:25:48Z",
    "message": "Feature \"members-switch-to-postgresdb\" is enabled for user \"Members-Accounts-live\"? false"
}
{
    "level": "info",
    "requestId": "a281f23f-0180-43e2-bf8d-50da98054f72",
    "sdkKey": "<sdk-key>",
    "time": "2024-09-15T15:25:48Z",
    "message": "Feature \"dummy-flag\" is enabled for user Members-Accounts-live? false"
}
..

OPTIMIZELY_SDKKEYS env var is set properly and matches the sdkKey appearing in the follow-up logs.
This also occurs on every message with "Loading Optimizely instance"

As you can see the first few entries will always error on unable to find SDK keys but after that it would work as expected.
Just wondering if this is an expected behaviour or we misconfigured something.
None of the developers notice an actual degradation on our services but it would be nice not to see these in our logs if things are working as expected.

Expected Behavior

Should not show up as logs if SDK keys are set correctly

Steps To Reproduce
  1. Deploy as a container/task on ECS
  2. Look at logs after a few minutes
Go Version

No response

Link

No response

Logs

No response

Severity

No response

Workaround/Solution

No response

Recent Change

No response

Conflicts

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the startup behavior in the ECS/Fargate container and trace the initialization and datafile-fetch logs when OPTIMIZELY_SDKKEYS is set. Done means valid SDK keys no longer produce misleading transient 403 errors during initialization while subsequent feature evaluations continue to work.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, go
Domain
backend, cloud
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.