pelias / pelias/api

Avoid repeated normalization of every alias pair in `diffPlaces`

Open Beginner friendly
#1,722 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
JavaScript
Stars
259
Forks
168
Avg merge
6h 15m
Merged PRs (30d)
1

Description

While testing address documents with many name aliases, duplicate checking became much more expensive than the Elasticsearch lookup. proposing a small optimization that preserves the existing duplicate decisions.

isPropertyDifferent compares every value on one side with every value on the other, normalizing both inside the nested loop. For disjoint arrays of lengths m and n, that performs O(m × n) normalization work for each property comparison.

In a NYC address result set I was using in local testing, 19 of 20 documents had 169 additional name aliases each. Comparing all 190 document pairs through isDifferent produced:

Implementation Elapsed time Pairs classified as different
Existing helper 30,587 ms 190
Normalize once and compare with a Set 387 ms 190

The proposed change is to normalize one array into a set, then normalize each value in the other array once and test membership. then return “same” on any overlap, retaining the current missing property and scalar to array handling. This makes the property-level comparison O(m + n), with O(n) temporary storage.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in helper/diffPlaces.js at isPropertyDifferent, especially lines 256-276, and trace its callers through isDifferent. Preserve the existing missing-property, scalar-to-array, and duplicate-decision behavior while avoiding repeated normalization; compare the result against the reported 190-pair behavior and performance improvement.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
backend, performance
Issue type
Refactor
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
86/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.