OpenFn / OpenFn/kit

Worker: logs don't get sent to GCP properly

Open
#1,043 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

DevX
Dominant language
TypeScript
Stars
21
Forks
23
Avg merge
1d 22h
Merged PRs (30d)
17

Description

Worker logs currently goto STDOUT and appear in GCP

But it would be great to send a better structure to GCP. I'd love to, for example, include a label for different components in the logger, or a label with the run id. It would be so useful to see the related run as metadata on all log lines.

What do we need to do here? First off the worker should probably log as JSON, so that the structure can be used by GCP.

Secondly there must be some kind of way we can take the stdout stream from the worker and wrap it up into a format that GCP can use. Where does this code go?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing how the worker writes logs to STDOUT and how those logs are consumed by GCP. Determine where structured JSON and run or component metadata belong; done means worker logs arrive in GCP with usable structure and related-run metadata.

Written by the indexing model from the issue text.

Assessment

Tech stack
google-cloud, typescript
Domain
backend, cloud, observability
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.