Factually incorrect and suggested scripts and files not existent
还没有人认领这个 Issue。
- 主要语言
- Shell
- 星标
- 11.2k
- 派生
- 1.9k
- 平均合并
- 14 小时 16 分钟
- 30 天内合并 PR
- 6
描述
Describe the bug
The model described steps to perform the data discovery and alert building based upon the current files existing in the folder. However, although the steps sound logical and positively feasible, the files such as render.py, compare.py. etc are not truly present. They are purely made up by the model, in addition, when assessing the details about the crosswalk from index to the datastream in elastic, although the actual file was there in .csv format, the suggested transition from the index to the stream mentioned by the model was not existing in reality. The name of the dataset is not real either.
Affected version
No response
Steps to reproduce the behavior
Here is the step-by-step practical guide using the (redacted)/ as our working example.
────────────────────
End-to-End Workflow Map
[ Step 1: Data Discovery & Crosswalk ]
│ (Find source index, target datastream & ECS fields)
▼
[ Step 2: ES|QL Query Formulation & Data Evidence ]
│ (Convert SPL → ES|QL & test in Jupyter / Dev Tools)
▼
[ Step 3: YAML Specification & JSON Rendering ]
│ (Write spec-*.yaml → run render_alert.py)
▼
[ Step 4: Dry-Run Diff & Push to Kibana ]
│ (compare_alerts.py → render_alert.py --push)
▼
[ Step 5: Master Index Confirmation ]
(Update ALERTS_INDEX.md status)
────────────────────
Step 1: Data Discovery & Crosswalk Mapping
Goal: Determine where the Splunk data lives in Elasticsearch.
- Inspect the original Splunk query (from redacted):
index=redacted apiName="inf-bnkng-party-physaccsprof-accsprofvecos-v1"
| where apiPath="redacted"
| where responseCode >= 400 - Find the target Data Stream:
Look up in data/Kibana_dashboard_objects/...CROSSWALK.csv :
• Splunk redacted redacted logs $\rightarrow$ redacted
• Dataset: redacted - Map the Fields:
• apiPath $\rightarrow$ redacted
• redacted $\rightarrow$ redacted
────────────────────
Step 2: ES|QL Formulation & Data Evidence
Goal: Formulate and test the query against live data to verify hits.
- ES|QL Query Formulation:
redacted - Test & Verify:
• Run in Kibana Discover $\rightarrow$ ES|QL tab or in a verification notebook ( redacted.md ).
• Confirmation Criteria: Ensure no field type errors and that the count calculation matches expected failure counts.
────────────────────
Step 3: Create redacted
Goal: Avoid writing bulky Kibana JSON by using human-readable YAML.
- Create or edit the YAML spec ( redacted):
rule_id: "auto"
name: "redacted"
tags: ["redacted"]
schedule_interval: "5m"
time_window:
size: 5
unit: "m"
esql: |
redacted
email:
to: ["redacted]
subject: "{{context.hits.0._source.labels.environment}} - redacted."
snow:
node: "redacted"
resource: "/"
metric_name: "redacted"
short_description: "redacted" - Execute Python Rendering Script:
────────────────────
Step 4: Diff & Push to Kibana
### Expected behavior
Accurate to the individual details mentioned in the response from the file names, contents and inferred information (at least logical instead of making it up). The model should have suggested to create the files, instead of making them up to mislead.
### Additional context
_No response_
贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
调研方向
首先在 copilot-cli 中复现已报告的工作流,并将生成的引用与 data/Kibana_dashboard_objects/...CROSSWALK.csv 和现有的验证 notebook 进行比较。检查模型如何呈现 render.py、compare.py、render_alert.py、compare_alerts.py 以及 dataset 名称。完成的标准是:生成的指导能够区分现有文件和数据与建议的文件,或明确标注创建步骤。
由索引模型根据 Issue 内容生成。
评估
- 技术栈
- elasticsearch, python, shell, yaml
- 领域
- ai, cli
- Issue 类型
- 缺陷
- 难度
- 5/5
- 预计耗时
- 一周以上
- 活跃度
- 活跃
- 描述清晰度
- 需要澄清
- 新手友好度
- 25/100