graphql / graphql/graphql-spec

[RFC] Add a complete set of GraphQL Document examples for stress testing parsers

Open
#954 11 comments 8 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
14.6k
Forks
1.2k
PR merge metrics
No merged PRs in 30d

Description

## TL;DR

It would be really nice to have a set of GraphQL Documents which
collectively express the entire GraphQL grammar. This would include
atypical constructions such as random insertions of comma characters
as whitespace.

## Problem Statement

The GraphQL query language is well specified with a formal grammar but
is large enough that there can be subtle edge cases when constructing
parsers, especially when the parser is hand written. The language also
continues to be extended and library developers must stay on top of
changes to the spec.

## Proposed Solution

In addition to the language spec, we should providee an official
collection of GraphQL Documents which covers the entire grammar
constructed in varying levels of term complexity.

This set of \`Golden Documents\` should also contain examples which
attempt to trip up the parser implementation. For example,
inconsistent use of commas or extra commas, complexities around block
strings, and ambiguous parses such as:

> type Query
>
> {
> foo: bar
> }

For every \`Golden Document\` representing a valid term we would provide
a [graphql-js AST](https://github.com/graphql/graphql-js/blob/main/src/language/ast.ts#L293). For those representing invalid terms we would
provide the errors in [GraphQL format](https://spec.graphql.org/draft/#sec-Errors).

Between the [GraphQL Spec](https://spec.graphql.org/October2021/) examples, the [graphql-js](https://github.com/graphql/graphql-js) test suite, and
the [nim-graphql](https://github.com/status-im/nim-graphql/tree/master/tests) test suite we have a large supply of test values. In
addition I would also like to build a set of terms based on the
[chinook-database](https://github.com/lerocha/chinook-database) generated from the [Hasura GraphQL Engine](https://github.com/hasura/graphql-engine).

Such a collection of Documents would be extremely useful for golden
testing GraphQL parsers and would help establish a higher standard for
parsers across implementation languages.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.