elastic / elastic/integrations

[tenable_sc] asset fingerprint uses identity fields only — state changes are silently dropped

Open
#21,251 1 comment 0 reactions 0 assignees View on GitHub
bug Integration:tenable_sc Team:Security-Service Integrations
Dominant language
Handlebars
Stars
333
Forks
647
Avg merge
3d 4h
Merged PRs (30d)
209

Description

The asset ingest pipeline computes a SHA-256 fingerprint from the fields listed in the Tenable.sc `uniqueness` field (e.g. `repositoryID,ip,dnsName`) and stores it in `tenable_sc.asset.custom_hash`, which is used as the document `_id`. Because those are identity fields, every mutable state field — `score`, `total`, all six severity counts, `macAddress`, `policyName`, `pluginSet`, `lastAuthRun`, `lastUnauthRun`, etc. — is excluded from the fingerprint.

The data stream is append-only. When the same asset is returned on the next poll with a changed severity count (e.g. a vulnerability has been remediated), the pipeline produces a document with the same `_id`. The data stream rejects it and the state change is never recorded in Elasticsearch.

The fingerprint was introduced in #2718 with no associated design discussion. The stated intent was "finding uniqueness in Assets", but the consequence of using an identity-only hash as `_id` in an append-only index was not addressed.

**Two fixes are possible, both requiring a latest transform:**

1. Extend the fingerprint to include all mutable state fields alongside the identity fields. The data stream becomes a time series of distinct observed states (bounded growth — unchanged assets still deduplicate). A latest transform keyed on the identity provides a current-state view.

2. Remove the fingerprint processor entirely and rely on auto-generated `_id`. Every document on every poll is indexed unconditionally. A latest transform is then unconditionally required to query current state.

Option 1 is preferable: it bounds data growth to genuine state changes and makes the transform advisory rather than mandatory for correctness.

Contributor guide

Open the contributing guide

Research direction

Start by tracing the Tenable.sc asset ingest pipeline and the fingerprint processor introduced in #2718, then inspect how the append-only data stream handles repeated _id values. Verify the chosen fix preserves changed asset states and that a latest transform provides the current-state view.

Written by the indexing model from the issue text.

Assessment

Tech stack
handlebars
Domain
data-engineering, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.