splunk / splunk/pytest-splunk-addon

Support testing timestamping that is applied using an eval during indexing

Open
#402 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
66
Forks
21
Avg merge
4d 13h
Merged PRs (30d)
2

Description

It seems to us that the framework is not really testing timestamp extraction as the _time test will always pass regardless of the sample and props&transforms.

Our sample:
{"RoleLocation":"South Central US","time":"##Timestamp##"}

Our transform:

INGEST_EVAL = _time=coalesce(strptime(spath(_raw, "time"), "%Y-%m-%dT%H:%M:%S.%QZ"),strptime(spath(_raw, "time"), "%m/%d/%Y %l:%M:%S %p"),_time)

Our test(token.0.replacement was intentionally set to a bad value to prove the test will always pass):

[azure_activity_log.sample]
host_type = plugin

input_type = modinput
index = main
sourcetype = azure:activity
sourcetype_to_search = azure:activity
sample_count = 1
expected_event_count = 1

timestamp_type = event

token.0.token = ##Timestamp##
token.0.replacementType = timestamp
token.0.replacement = a
token.0.field = _time

earliest = -30m
latest = -30m

See picture for Ingested event with correct timestamp(now-30min) specified by test but neither is in sample nor ingest time.
image
No props&transforms were added when testing.
All tests passed

@rfaircloth-splunk 's suggestion is to have a new timestamp_type that:

this would need an option for "none" or "random" so that the even level ts is either not sent or sent invalid to confirm the eval itself works

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No file or test path is named; start by tracing handling of timestamp_type through the indexing test flow and how props and transforms are applied. Done should include a none or random timestamp option so an eval-derived _time cannot pass because of a supplied event timestamp, with the intentionally bad replacement exposing failures.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing-qa
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.