bee-san / bee-san/pyWhat

Add support for non-regex processing

Open
#115 0 comments 3 reactions 0 assignees View on GitHub
enhancement help wanted
Dominant language
Python
Stars
7.3k
Forks
391
PR merge metrics
No merged PRs in 30d

Description

Currently, pywhat only uses regular expressions to identify stuff. However, we may often want to update the description based on a match or to filter out false positives.

Some examples:

- Pinging URLs to check if it is responding or if a certain resource exists, etc.
- Parsing timestamps (https://github.com/bee-san/pyWhat/issues/234) (`1637093119.558717` to `November 16, 2021 8:05:19 PM`)
- Changing rarity based on a match
- And many more

How it should be implemented?

I think the best idea would be to write a class for every regex pattern (so that it can save the state between several matches). That class should have a method like `process()` that gets a match object and can alter it and return it or return None to filter it out.

Example:

```py
class URLProcessor:
def process(match):
if "http://trashurl.it" in match["url"]: # Or maybe we should create an actual Match class and do match.url
return None
```

Contributor guide

Open the contributing guide

Research direction

No files, tests, or entry points are named. Start by tracing how pywhat currently applies regular expressions and represents matches, then compare that flow with the proposed stateful processor and Match interface; done should support URL checks, timestamp parsing, metadata changes, and filtering false positives without breaking existing regex identification.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.