rebalance fallback.venue queries
Nobody has claimed this yet.
- Dominant language
- JavaScript
- Stars
- 259
- Forks
- 169
- Avg merge
- 6h 15m
- Merged PRs (30d)
- 1
Description
I was testing https://github.com/pelias/api/pull/1380 today and found this bug related to venue queries with the /v1/search API.
Given the following parse:
"parser": "libpostal",
"parsed_text": {
"query": "target",
"city": "eureka",
"state": "ca"
}
The following query is generated (pasted at the end for brevity)
We can see that many results were returned from ES:
{
"search_fallback":{
"es_took":229,
"response_time":233,
"retries":0,
"es_hits":44,
"es_result_count":20
}
}
but the results are:
0) Eureka, CA, USA
1) Plumas Eureka, CA, USA
when they should be:
1) Target, Eureka, CA, USA
When I ran the ES query manually I found that all of the top results are coming from the admin layers:
"fallback.region"
"fallback.locality"
"fallback.locality"
"fallback.region"
"fallback.locality"
"fallback.locality"
"fallback.locality"
"fallback.region"
"fallback.region"
"fallback.region"
"fallback.region"
"fallback.region"
"fallback.region"
"fallback.region"
"fallback.region"
"fallback.region"
"fallback.region"
"fallback.region"
"fallback.region"
"fallback.region"
What I suspect is happening here is that the population scoring is increasing the scores for administrative places much higher than those for venues, meaning that the venue results are not returned.
We can probably fix this by applying a boost to the fallback.venue clause so that it can compete with the high scores generated by admin matches:
# scores corresponding to the admin fallback matches listed directly above
"_score": 47.895332,
"_score": 44.888824,
"_score": 44.842762,
"_score": 39.021828,
"_score": 22.350563,
"_score": 22.343859,
"_score": 22.316488,
"_score": 19.958286,
"_score": 19.931473,
"_score": 16.801222,
"_score": 16.78785,
"_score": 16.157461,
"_score": 16.063375,
"_score": 15.326169,
"_score": 15.228479,
"_score": 15.188125,
"_score": 15.156264,
"_score": 15.089676,
"_score": 15.087963,
"_score": 15.082789,
{
"function_score":{
"query":{
"bool":{
"minimum_should_match":1,
"should":[
{
"bool":{
"_name":"fallback.venue",
"must":[
{
"multi_match":{
"query":"target",
"type":"phrase",
"fields":[
"phrase.default"
]
}
},
{
"multi_match":{
"query":"eureka",
"type":"phrase",
"fields":[
"parent.locality",
"parent.locality_a",
"parent.localadmin",
"parent.localadmin_a"
]
}
},
{
"multi_match":{
"query":"ca",
"type":"phrase",
"fields":[
"parent.region",
"parent.region_a",
"parent.macroregion",
"parent.macroregion_a"
]
}
}
],
"filter":{
"term":{
"layer":"venue"
}
}
}
},
{
"bool":{
"_name":"fallback.locality",
"must":[
{
"multi_match":{
"query":"eureka",
"type":"phrase",
"fields":[
"parent.locality",
"parent.locality_a"
]
}
},
{
"multi_match":{
"query":"ca",
"type":"phrase",
"fields":[
"parent.region",
"parent.region_a",
"parent.macroregion",
"parent.macroregion_a"
]
}
}
],
"filter":{
"term":{
"layer":"locality"
}
}
}
},
{
"bool":{
"_name":"fallback.localadmin",
"must":[
{
"multi_match":{
"query":"eureka",
"type":"phrase",
"fields":[
"parent.localadmin",
"parent.localadmin_a"
]
}
},
{
"multi_match":{
"query":"ca",
"type":"phrase",
"fields":[
"parent.region",
"parent.region_a",
"parent.macroregion",
"parent.macroregion_a"
]
}
}
],
"filter":{
"term":{
"layer":"localadmin"
}
}
}
},
{
"bool":{
"_name":"fallback.region",
"must":[
{
"multi_match":{
"query":"ca",
"type":"phrase",
"fields":[
"parent.region",
"parent.region_a"
]
}
}
],
"filter":{
"term":{
"layer":"region"
}
}
}
},
{
"bool":{
"_name":"fallback.macroregion",
"must":[
{
"multi_match":{
"query":"ca",
"type":"phrase",
"fields":[
"parent.macroregion",
"parent.macroregion_a"
]
}
}
],
"filter":{
"term":{
"layer":"macroregion"
}
}
}
}
],
"filter":{
"bool":{
"must":[
{
"terms":{
"layer":[
"venue",
"street",
"country",
"macroregion",
"region",
"county",
"localadmin",
"locality",
"borough",
"neighbourhood",
"continent",
"empire",
"dependency",
"macrocounty",
"macrohood",
"microhood",
"disputed",
"postalcode",
"ocean",
"marinearea"
]
}
}
]
}
}
}
},
"max_boost":20,
"functions":[
{
"field_value_factor":{
"modifier":"log1p",
"field":"popularity",
"missing":1
},
"weight":1
},
{
"field_value_factor":{
"modifier":"log1p",
"field":"population",
"missing":1
},
"weight":2
}
],
"score_mode":"avg",
"boost_mode":"multiply"
}
}
note: the feature branch will unlikely still be on dev when you click this link ;)
https://pelias.github.io/compare/#/v1/search?text=Target+Eureka+CA&debug=1
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Begin at the /v1/search entry point and inspect the fallback query, especially the fallback.venue clause and population scoring. Reproduce the Target Eureka CA case with debug output; done means the venue is ranked ahead of administrative results without regressing other fallback layers.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- elasticsearch, javascript
- Domain
- api, backend, search
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100