ankane / ankane/pdscan

Not working with elasticsearch aliases

Open
#19 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
283
Forks
37
PR merge metrics
No merged PRs in 30d

Description

Hi, I'm using filtered aliases to segment the data and they should work like indices , but for some reason are being ignored.

Go to Kibana Devtools and run the following:

DELETE test_animals

POST test_animals/_doc
{
  "name": "Peter Parker",
  "type": "dog"
}
POST test_animals/_doc
{
  "name": "Michael Jordan",
  "type": "cat"
}
POST _aliases 
{
  "actions": [
    {
      "add": {
        "index": "test_animals",
        "alias": "dogs",
        "filter": {
          "term": {
            "type.keyword": "dog"
          }
        }
      }
    },
    {
      "add": {
        "index": "test_animals",
        "alias": "cats",
        "filter": {
          "term": {
            "type.keyword": "cat"
          }
        }
      }
    }
  ]
}

This will generate two virtual indices based on one original indices, giving the the ability to run pdscan against segments of the data:

./pdscan elasticsearch+https://gustavo:gustavo@my-deployment-xxx:9200/cats --show-all --format ndjson --show-data
It should report only "michael jordan" but reports against all test_animals index.

Found 1 index to scan, sampling 10000 documents from each...

{"identifier":"test_animals.name","name":"surname","match_type":"value","confidence":"low","matches":["Michael Jordan","Peter Parker"],"matches_count":2}

Is this expected? how can I segment by a certain field?

Thanks!

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the alias setup in Kibana DevTools, then run the shown pdscan command against the cats alias and trace the Elasticsearch scan entry point handling that URL. Done means the cats scan reports only Michael Jordan rather than documents from test_animals.

Written by the indexing model from the issue text.

Assessment

Tech stack
elasticsearch, go
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.