mapbox / mapbox/mapbox-gl-js

Refactor benchmark taxonomy

Open
#4,216 0 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

testing :100:
Dominant language
TypeScript
Stars
12.4k
Forks
2.4k
PR merge metrics
No merged PRs in 30d

Description

This ticket proposes a new taxonomy of benchmarks for GL JS which aspires to be comprehensive, based on measurements directly relevant to our end users, and conceptually neat.

# Performance Metrics

- time-to-`dataend` against {Chrome, Firefox, Safari, IE11, Edge}, several styles, several camera positions, and {initial style load, `#setFilter` operation, small `setData` operation, large `setData` operation}
- {95th precentile frame time, 95th percentile frame duration, mean frame time, mean frame duration, mean framerate} against {Chrome, Firefox, Safari, IE11, Edge}, several styles, several camera positions, and {panning, rotating, pitching, zooming within an integer zoom level, zooming across integer zoom levels}
- memory usage against {Chrome, Firefox, Safari, IE11, Edge}, several styles, several camera positions, and {panning, rotating, pitching, zooming within an integer zoom level, zooming across integer zoom levels}
- number of GL API calls against {Chrome, Firefox, Safari, IE11, Edge}, several styles, several camera positions, and {panning, rotating, pitching, zooming within an integer zoom level, zooming across integer zoom levels}
- power usage against {Chrome, Firefox, Safari, IE11, Edge}, several styles, several camera positions, and {panning, rotating, pitching, zooming within an integer zoom level, zooming across integer zoom levels} (we could create a lightweight native app with a webview to measure this)

# Other Metrics

* test coverage
* statements
* branches
* functions
* lines
* flow type coverage
* compressed & gziped bundle size
* CI build time
* codebase size (we can use nyc for this too)
* statements
* branches
* functions
* lines
* number of flattened npm dependencies
* score measuring platform parity (GL JS vs GL Native vs Leaflet)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The issue names no files, tests, or benchmark entry points. Begin by locating the existing GL JS benchmark infrastructure and comparing it with the listed performance and project metrics; completion would require an agreed, implementable taxonomy.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
performance, testing-qa
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.