epfl-dlab / epfl-dlab/transformers-CFG

add support for input dependent grammar

Open
#14 0 comments 0 reactions 1 assignee Claimed by @Saibo-creator View on GitHub
enhancement
Dominant language
Python
Stars
139
Forks
23
PR merge metrics
No merged PRs in 30d

Description

The idea is to allow user to pass a list of grammar strings for a batch of inputs. And each grammar, input pair can be properly constrained, which is not supported in the current verison.

For the moment, regardless of batch size, the same batch can only be constrained with a single grammar object, which is equivalent to input-independent grammar.

In real world, function calls and json schema are great example where this is very useful. Now the two workarounds are:
1. use batch size = 1, which will be very inefficient for GPU
2. try to combine the grammars in a same batch to a more tolerent super set of each individual grammar, which is not already feasible and are relaxing the constraints.

To start this feature, a nice dataset is PennTree Bank where constraints can be input dependent.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.