epfl-dlab / epfl-dlab/transformers-CFG
add support for input dependent grammar
- Dominant language
- Python
- Stars
- 139
- Forks
- 23
- PR merge metrics
- No merged PRs in 30d
Description
The idea is to allow user to pass a list of grammar strings for a batch of inputs. And each grammar, input pair can be properly constrained, which is not supported in the current verison.
For the moment, regardless of batch size, the same batch can only be constrained with a single grammar object, which is equivalent to input-independent grammar.
In real world, function calls and json schema are great example where this is very useful. Now the two workarounds are:
1. use batch size = 1, which will be very inefficient for GPU
2. try to combine the grammars in a same batch to a more tolerent super set of each individual grammar, which is not already feasible and are relaxing the constraints.
To start this feature, a nice dataset is PennTree Bank where constraints can be input dependent.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.