ipfs / ipfs/kubo

HTTP APIs that accept binary data as query string args

Open
#7,958 10 comments 0 reactions 1 assignee Claimed by @lidel View on GitHub
kind/enhancement status/ready topic/rpc-api
Dominant language
Go
Stars
17.1k
Forks
3.2k
Avg merge
3d 18h
Merged PRs (30d)
11

Description

#7939 exposed two issues with the HTTP API. A solution for one was [proposed](https://github.com/ipfs/go-ipfs/issues/7939#issuecomment-788088091), the other was discussed at triage and I wanted to capture the scope of the work here.

Some of our API endpoints accept query string arguments that contain strings which will be interpreted as binary data. This causes problems for browsers and other JS clients in that the built in URL encoding functions accept strings and do UTF8 multi-byte code point conversions for byte values above `127`/`0x7F`, so `0x80` becomes `%EF%BF%BD` instead of `%80`.

Instead we should allow sending [multibase](https://github.com/multiformats/multibase) encoded strings for these arguments removing any ambiguity over their contents. The proposal was to add an additional flag that indicates to the API that the received string should be interpreted as a multibase encoded string and not a byte array.

I've done a quick pass over the [HTTP API docs](https://docs.ipfs.io/reference/http/api/) and these endpoints currently interpret an argument as a byte array, the good news is there aren't many:

`/api/v0/dht/findpeer` - `arg` needs encoding
`/api/v0/dht/findprovs` - `arg` needs encoding
`/api/v0/dht/provide` - `arg` needs encoding
`/api/v0/dht/put` - `arg` needs encoding
`/api/v0/dht/query` - `arg` needs encoding
`/api/v0/pubsub/pub` - second `arg` needs encoding (also accepts payload in message body?)

These endpoints accept arguments that by convention would be B58 encoded multibase strings containing CIDs, but it is not explicit in the docs so could use a note saying the value `'should be an IPFS path or multibase encoded string containing a CID'` or similar depending on context.

`/api/v0/filestore/ls` - `arg` may need encoding note?
`/api/v0/filestore/verify` - `arg` may need encoding note?
`/api/v0/id` - `arg` may need encoding note?
`/api/v0/name/resolve` - `arg` may need encoding note?
`/api/v0/object/diff` - `arg` may need encoding note?
`/api/v0/object/patch/add-link` - `arg` may need encoding note?
`/api/v0/object/patch/append-data` - `arg` may need encoding note?
`/api/v0/object/patch/rm-link` - `arg` may need encoding note?
`/api/v0/object/patch/set-data` - `arg` may need encoding note?
`/api/v0/ping` - `arg` may need encoding note?
`/api/v0/stats/bw` - `peer` may need encoding note?
`/api/v0/swarm/connect` - `arg` may need encoding note?
`/api/v0/swarm/disconnect` - `arg` may need encoding note?

There may be other affected endpoints.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.