webtorrent / webtorrent/node-bencode

`node-bencode` can produce dictionary entries with duplicate keys.

Open
#146 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
JavaScript
Stars
173
Forks
38
PR merge metrics
No merged PRs in 30d

Description

Bug

node-bencode can produce dictionary entries with duplicate keys.


node-bencode assumes that binary string keys made out of unique Javascript string keys are unique as well, which is false.

https://github.com/webtorrent/node-bencode/blob/ee70f267c8d34b9a94820ca8c42cd67d1274fc89/lib/encode.js#L53-L55
https://github.com/ThaUnknown/uint8-util/blob/149c44c010b3ad17a7904c4266545bbca1fd4403/_node.js#L13

 encode.string = function (buffers, data) { 
   buffers.push(text2arr(text2arr(data).byteLength + ':' + data)) 
 } 
export const text2arr = str => new Uint8Array(Buffer.from(str, 'utf8'))

Proof-of-concept

For example, let node-bencode try encoding {"\uD800": 1, "\uDFFF": 2}. It’ll produce dictionary entries with the duplicate key, "3:\xEF\xBF\xBD".

const lone_surrogates = "\uD800\uDFFF";
// Lone (“unmatched”) UTF-16 surrogates. Invalid in UTF-16.

const a = Buffer.from(lone_surrogates[0], "UTF-8");
const b = Buffer.from(lone_surrogates[1], "UTF-8");
// Decoding the Javascript strings in UTF-16 and encoding them into UTF-8.

console.log(a, a.toString(), b, b.toString());
//  Since those Javascript strings are invalid in UTF-16,
// those lone surrogates are decoded
// into `REPLACEMENT CHARACTER`s (U+FFFD)
// and subsequently encoded into `<Buffer ef bf bd>`.
// Meaning,

console.log(a.equals(b));
// is true, when (lone_surrogates[0] === lone_surrogates[1]) is false.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with lib/encode.js lines 53-55 and the linked uint8-util _node.js implementation; run the provided lone-surrogate proof of concept. Trace how the JavaScript keys become UTF-8 bytes and add a regression check for the reported object. Done means encoding it no longer emits dictionary entries with duplicate keys.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, node.js
Domain
backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.