Filtered Mapbox Clusters Appear in Incorrect Locations When Using Gzipped GeoJSON
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 12.4k
- Forks
- 2.4k
- PR merge metrics
- No merged PRs in 30d
Description
We have a large dataset (~800k items) that we are rendering with Mapbox clusters. We have also precomputed our assets and stored them in cloud storage - as gzipped geojson files.
To allow users "Apply Filters" to the data, we make a request to our backend endpoint which then computes the relevant items that match the filters.
Since this is a lighter operation, our backend endpoint sends the results to the frontend as a gzip file. And this is where our issue comes in. Even though we use the same methods for both operations, when we use our backend to compute and return the gzip file, some points and clusters are rendered at very odd locations on the map - sometimes in the ocean.
While the data is accurate (as expected), the points are simply rendered at wrong locations on the map. Pls have a look at the provided screenshots.
How it works:
To illustrate this issue, I'm providing sample datasets and screenshots.
Dataset 1: Large dataset. Renders properly as expected.
Dataset 2: Filtered dataset. Does not render/cluster as expected. Has a couple of odd points and clusters.
This is a link to a problematic dataset: https://drive.google.com/file/d/1jOxxtnBTbKeUjuNtj5CWS_PTYLBjlYFG/view?usp=sharing
Here's an excerpt of the React code that renders the map:
```
const mapSrcName = options.isFiltered ? "filtered-properties" : "properties";
const mapSrcData = options.isFiltered ? insightsData.filters.items : (process.env.NEXT_PUBLIC_MAP_SRC_URL as string);
const mapClusterOptions = {
cluster: true,
clusterMaxZoom: 16,
clusterRadius: 50,
};
newMap.addSource(mapSrcName, {
type: "geojson",
data: mapSrcData,
generateId: true,
...mapClusterOptions,
});
```
Here's how we compute and return data via our backend endpoint:
```
async filterInsights(filterOptions: any) {
const parsedFilterOptions = JSON.parse(filterOptions);
if (parsedFilterOptions.location) {
const response = await fetch(`https://${CLOUD_URL}/precomputed-MN.geojson.gz`,
);
const boroughFeatureCollection = await response.json();
const boroughFeaturesOfInterest = (
boroughFeatureCollection.features ?? []
).filter((f) => {
// our custom filter checker
checkFeatureMatchesFilters(f, parsedFilterOptions),
});
const newBoroughFC = {
...boroughFeatureCollection,
features: boroughFeaturesOfInterest,
};
// here we create our feature collections object
const clusteredFC = getInsightClusterFeatureCollection(newBoroughFC.features);
// here we create a stream which we then pass to the client as gzip
const collectionStream = createFeatureCollectionStream(clusteredFC);
return collectionStream.pipe(zlib.createGzip({ level: 9 }));
}
return null;
}
```
Here is the method that creates the stream that's used in applyFilters method above:
```
createFeatureCollectionStream(featureCollection: any) {
return new Stream.Readable({
read() {
// Start JSON
this.push('{"type": "FeatureCollection", "features": [');
featureCollection.features.forEach((feature, index) => {
this.push(
JSON.stringify(feature) +
(index < featureCollection.features.length - 1 ? ',' : ''),
);
});
this.push(']}'); // End JSON
this.push(null); // End Stream
},
});
}
```
Any insights into this issue would be appreciated.
**mapbox-gl-js version**: 2.15.0
**browser**: Firefox, Chrome
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the issue with the linked problematic dataset and the Mapbox GL JS 2.15.0 setup. Compare the precomputed GeoJSON source with the filtered data produced by filterInsights, including createFeatureCollectionStream and gzip handling. Done means filtered points and clusters render at their correct locations.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- react, typescript
- Domain
- frontend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100