gchq / gchq/CyberChef

Feature request: sequential pattern mining

Open
#1,195 0 comments 0 reactions 0 assignees View on GitHub
feature
Dominant language
JavaScript
Stars
35.8k
Forks
4.1k
Avg merge
2d 26m
Merged PRs (30d)
33

Description

Sequential pattern mining is a general data mining solution for finding patterns in sequences e.g. if you have a text something like
```
ABCD
CBCD
BECD
```
then it can find the `BCD` as a frequent closed sequence with the support of 3. It is good to prepare the data first, e.g. in the case of logs split up to words and give each word an individual id and each special character an individual id, use the algorithm on the resulting array and do reverse id -> word mapping on the results. This is necessary to spare CPU and memory, otherwise it would eat a lot of resources. Different type of data might need different preparation. SPM can be used for objects with different properties too and the algorithms can check multiple properties, not just a single one, so in the case of JSON, not plain text this can be an extra feature.

I don't think there is any alternative tool for this. There are countless algorithms for this, I don't know which would be the best, definitely not the old ones like GSP or SPADE. They might need a different thread, otherwise it might freeze the window.

Contributor guide

Open the contributing guide

Research direction

The issue names no files, tests, or entry points; start by reviewing CyberChef's existing operation structure and how data-analysis features are tested. Before implementation, define the sequential pattern mining algorithm, supported input and output formats, resource limits, and acceptance tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.