nodejs / nodejs/undici

"preprocessed" headers

Open
#3,998 8 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement Status: help-wanted
Dominant language
JavaScript
Stars
7.7k
Forks
880
Avg merge
2d 16h
Merged PRs (30d)
68

Description

When using the same headers for multiple requests it would be nice to bypass much of the header validation and transformation logic we do:

i.e. an API like this should be possible:

const headers = new undici.Headers({ Foo: 'bar' })

for (let n = 0; n < 100; n++) {
  const res = await request(url, { headers })
  await res.dump()
}

Where undici.Headers guarantee that the headers are already in a valid state and we don't need to do anything further. Could even be pre computed into a headers string.

To start with I would makeundici.Headers immutable. Later we could add immutable composition, e.g:

const headers = new undici.Headers({ Foo: 'bar' })

for (let n = 0; n < 100; n++) {
  const res = await request(url, { headers: headers.assign({ Bar: 'foo' }) })
  await res.dump()
}

A quick start (just move logic from core/request constructor and make request access the state through symbols):

class Headers {
  constructor(headers) {
    if (Array.isArray(headers)) {
      if (headers.length % 2 !== 0) {
        throw new InvalidArgumentError('headers array must be even')
      }
      for (let i = 0; i < headers.length; i += 2) {
        processHeader(this, headers[i], headers[i + 1])
      }
    } else if (headers && typeof headers === 'object') {
      if (headers[Symbol.iterator]) {
        for (const header of headers) {
          if (!Array.isArray(header) || header.length !== 2) {
            throw new InvalidArgumentError('headers must be in key-value pair format')
          }
          processHeader(this, header[0], header[1])
        }
      } else {
        const keys = Object.keys(headers)
        for (let i = 0; i < keys.length; ++i) {
          processHeader(this, keys[i], headers[keys[i]])
        }
      }
    } else if (headers != null) {
      throw new InvalidArgumentError('headers must be an object or an array')
    }
  }
}


function processHeader (headers, key, val) {
  if (val && (typeof val === 'object' && !Array.isArray(val))) {
    throw new InvalidArgumentError(`invalid ${key} header`)
  } else if (val === undefined) {
    return
  }

  let headerName = headerNameLowerCasedRecord[key]

  if (headerName === undefined) {
    headerName = key.toLowerCase()
    if (headerNameLowerCasedRecord[headerName] === undefined && !isValidHTTPToken(headerName)) {
      throw new InvalidArgumentError('invalid header key')
    }
  }

  if (Array.isArray(val)) {
    const arr = []
    for (let i = 0; i < val.length; i++) {
      if (typeof val[i] === 'string') {
        if (!isValidHeaderValue(val[i])) {
          throw new InvalidArgumentError(`invalid ${key} header`)
        }
        arr.push(val[i])
      } else if (val[i] === null) {
        arr.push('')
      } else if (typeof val[i] === 'object') {
        throw new InvalidArgumentError(`invalid ${key} header`)
      } else {
        arr.push(`${val[i]}`)
      }
    }
    val = arr
  } else if (typeof val === 'string') {
    if (!isValidHeaderValue(val)) {
      throw new InvalidArgumentError(`invalid ${key} header`)
    }
  } else if (val === null) {
    val = ''
  } else {
    val = `${val}`
  }

  if (request.host === null && headerName === 'host') {
    if (typeof val !== 'string') {
      throw new InvalidArgumentError('invalid host header')
    }
    // Consumed by Client
    headers[kHost] = val
  } else if (request.contentLength === null && headerName === 'content-length') {
    headers[kContentLength] = parseInt(val, 10)
    if (!Number.isFinite(request.contentLength)) {
      throw new InvalidArgumentError('invalid content-length header')
    }
  } else if (request.contentType === null && headerName === 'content-type') {
    headers[kContentType] = val
    headers[kHeaders].push(key, val)
  } else if (headerName === 'transfer-encoding' || headerName === 'keep-alive' || headerName === 'upgrade') {
    throw new InvalidArgumentError(`invalid ${headerName} header`)
  } else if (headerName === 'connection') {
    const value = typeof val === 'string' ? val.toLowerCase() : null
    if (value !== 'close' && value !== 'keep-alive') {
      throw new InvalidArgumentError('invalid connection header')
    }

    if (value === 'close') {
      headers[kReset] = true
    }
  } else if (headerName === 'expect') {
    throw new NotSupportedError('expect header not supported')
  } else {
    request[kHeaders].push(key, val)
  }
}

Refs: https://github.com/nodejs/undici/issues/3994

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the header validation and transformation logic in the core/request constructor, then inspect the proposed Headers and request entry points. The work is complete when validated headers can be reused across requests without repeating that processing, while preserving the documented validation and special-header behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
api, backend
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.