`seqcli ingest` extract embedded JSON values from plain text logs
Nobody has claimed this yet.
- Dominant language
- C#
- Stars
- 167
- Forks
- 30
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 9
Description
Discussed in https://github.com/datalust/seq-tickets/discussions/1536
Originally posted by mkvonarx March 31, 2022
Hi
We're using serilog for our logging. Unfortunately, we cannot user the CLEF JSON format in production. Instead we use a more human readable format with # to delimit the values and a {Properties} at the end to also output all remaining Serilog properties. Our format string looks almost like this:
"outputTemplate": "{Timestamp:yyyy-MM-dd HH:mm:ss.fff} # {ThreadId,3} # {Level:u3} # {SourceContext,-30} # {Message:lj} # {Properties} # {Exception}${NewLine}"
We're trying to ingest these logs to Seq with "seqcli ingest -x" and it works mostly fine. The only thing we cannot manage to nicely ingest is the {Properties} part. This is a JSON inside the human readable text, and seqcli ingest does not have a JSON parser according to https://docs.datalust.co/docs/command-line-client#extraction-patterns
Is there any way to ingest the {Properties} with seqcli ingest as anything else than a string?
Having a 5th builtin property like e.g. @p that directly understands JSON like {Properties} (like @t etc.) would propably be the nicest solution. Or maybe something like "seqcli ingest -x ".... {:json} ..." could also work.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the seqcli ingest -x extraction-pattern behavior and the linked command-line client documentation. The change should let embedded JSON in plain-text log properties be ingested as structured values rather than a string; the payload does not identify implementation files or tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- cli
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100