nodejs / nodejs/undici

feat: preparsed Headers

Open
#3,582 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
JavaScript
Stars
7.7k
Forks
880
Avg merge
2d 16h
Merged PRs (30d)
68

Description

Let's rewrite the manipulation of request/response headers already?

In undici, [string, string][] for headers are passed almost everywhere, and almost everywhere it is checked and reduced to the same type

Maybe it's worth writing a Headers List and using it everywhere inside the library?

In client-1 client-2, you can immediately set the headersList: new Headers List() and then use this structure everywhere

This structure can be used for all Compose Interceptors and for Handlers

In the current version, only fetch is converted from any type to Record<string, string>, and request just passes this raw data on

My motivation is that I need to make sure that the interceptors and handlers always have the data structure we need and then only transfer it, and not parse Buffer[] in each interceptor or handler, make a modification and throw it further.

1) create request/fetch -> anyHeaders to HeadersList
2) call dispatcher compose interceptors
3) call dispatcher.dispatch

4) llhttp onStatus, create HeadersList()
5) llhttp onHeaderName + onHeaderValue -> HeadersList.append(name, value);
6) call handlers events handler.onHeaders(status, HeadersList, ...)

You can make a separate interceptor that would lead from any type of headers to the Headers List, and would do the same with onHeaders. But do we even need raw headers in the form of Buffer[]?

const kRawHeadersMap = Symbol('raw headers map');

export interface IHeadersList {
	[kRawHeadersMap]: Map<string, string[]>;

	set(name: string, value: string): void;
	append(name: string, value: string): void;

	get(name: string): string | null;
	has(name: string): boolean;
	delete(name: string): boolean;

	keys(name: string): string[];
	values(name: string): string[];

	getCookies(): string[];
	getSetCookies(): string[];

	entries(): IterableIterator<[string, string]>;
	[Symbol.iterator](): IterableIterator<[string, string]>;
}


export class HeadersList implements IHeadersList {
	[kRawHeadersMap]: Map<string, string[]> = new Map();
	
	get size(){
		return this[kRawHeadersMap].size;
	}
	
	set(name: string, value: string): void {
		this[kRawHeadersMap].set(name, [value]);
	}

	append(name: string, value: string): void {
		const values = this[kRawHeadersMap].get(name);

		if (values !== undefined) values.push(value);
		else this[kRawHeadersMap].set(name, [value]);
	}

	get(name: string): string | null {
		const values = this[kRawHeadersMap].get(name);

		if (values !== undefined) return values[0];
		else return null;
	}

	has(name: string): boolean {
		return this[kRawHeadersMap].has(name);
	}

	delete(name: string): boolean {
		return this[kRawHeadersMap].delete(name);
	}

	keys(name: string): string[] {
		if (this[kRawHeadersMap].size === 0) return [];

		name = name.toLowerCase();

		const keys: string[] = [];
		for (const key of this[kRawHeadersMap].keys()) {
			if (name === key.toLowerCase()) keys.push(key);
		}

		return keys;
	}

	values(name: string): string[] {
		return this[kRawHeadersMap].get(name) ?? [];
	}

	getCookies(): string[] {
		const cookies: string[] = [];

		for (const name of this.keys('cookie')) {
			const values = this.values(name);
			cookies.push(...values);
		}

		return cookies;
	}

	getSetCookies(): string[] {
		const setCookies: string[] = [];

		for (const name of this.keys('set-cookie')) {
			const values = this.values(name);
			setCookies.push(...values);
		}

		return setCookies;
	}

	*entries(): IterableIterator<[string, string]> {
		for (const [name, values] of this[kRawHeadersMap].entries()) {
			const lowerCasedName = name.toLowerCase();

			// set-cookie
			if (lowerCasedName === 'set-cookie') {
				for (const value of values) yield [name, value];
			}
			// one value
			else if (values.length === 1) yield [name, values[0]];
			// more values
			else {
				const separator = lowerCasedName === 'cookie' ? '; ' : ', ';
				yield [name, values.join(separator)];
			}
		}
	}

	rawKeys(): IterableIterator<string> {
		return this[kRawHeadersMap].keys();
	}

	rawValues(): IterableIterator<string[]> {
		return this[kRawHeadersMap].values();
	}

	rawEntries(): IterableIterator<[string, string[]]> {
		return this[kRawHeadersMap].entries();
	}

	[Symbol.iterator](): IterableIterator<[string, string]> {
		return this.entries();
	}

	toArray() {
		return Array.from(this.entries());
	}

	toObject() {
		return Object.fromEntries(this.entries());
	}
}
// @ts-expect-error:
const setTimestampInterceptor = dispatch => (opts, handler) => {
	const headers = opts.headers as IHeadersList;

	// request headers list names must be all cased!
	// x-ts-date
	// X-Ts-Date
	// x-TS-date
	headers.set('X-TS-Date', `${(Date.now() / 1000 - 1e3).toFixed(0)}`);
	headers.set('X-ts-Date', `${(Date.now() / 1000 + 1e3).toFixed(0)}`);

	class MyHandler extends DecoratorHandler {
		onRawHeaders(
			status: number,
			headers: Buffer[],
			resume: () => void,
			statusText?: string
		) {}

		onHeaders(
			status: number,
			headersList: IHeadersList,
			resume: () => void,
			statusText?: string
		) {
			// response headers names is all lower cased
			if (headersList.has('content-encoding')) {
				//
			}

			return super.onHeaders(status, headersList, resume, statusText);
		}
	}

	return dispatch(opts, new MyHandler(handler));
};

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Trace request/fetch through the dispatcher compose interceptors and dispatcher.dispatch, then follow llhttp's onStatus, onHeaderName, and onHeaderValue into the handlers. Done would mean the proposed HeadersList flow is consistently defined for request and response headers, including interceptor and handler behavior, without repeated raw Buffer[] parsing.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, typescript
Domain
backend-api-design
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.