protomaps / protomaps/PMTiles

.pmtiles files can be any unicode char but aws/cloudflare only accept S3 safe characters

Open
#490 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
TypeScript
Stars
3k
Forks
196
Avg merge
19h 2m
Merged PRs (30d)
5

Description

In the AWS and Cloudflare proxy implementations we use a regex to parse the URL to a object storage key https://github.com/protomaps/PMTiles/blob/main/serverless/shared/index.ts#L8

This only allows S3-safelisted characters https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-keys.html

But this means that if I have a valid but non safe listed filename我的城市.pmtiles in s3, then I put the lambda proxy in front, it will not work, it will return 404.

We should accept any character in URLs but be careful to avoid path traversal attacks, etc.

This also affects go-pmtiles and thus the GCP and Azure implementations: https://github.com/protomaps/go-pmtiles/blob/main/pmtiles/server.go#L441

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the regex in serverless/shared/index.ts and the handler around pmtiles/server.go:441, then trace how AWS, Cloudflare, GCP, and Azure implementations derive object-storage keys from URLs. Confirm the behavior for the Unicode filename example and define handling that accepts valid URL characters while rejecting path traversal; done means all affected proxies resolve such keys safely instead of returning 404.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, typescript
Domain
backend, cloud
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.