Prompt Template for Code Synthesis

Separate into "prompt template" (a GPT-3 recognized word) and variables. Then ask the gpt-3 to change specific parameters for the template.

Prompt template: What is the [variable] of the [object]?

- variable-object pairs = (currency, Myanmar), (price, Bitcoin)
Now, for generation we might specify differences for either prompt templates or variable/object. Based on following parameters(not comprehensive list):

i. Style: Keep the meaning of the question same but ask in different styles.
eg: introduce spelling mistakes, code mixing, cultural tone difference etc.

ii. Semantic diversity: The task itself should change.
Add the above nlp metrics in code and set thresholds, keep generating till the thresholds are satisfied.

Reference:
1. https://arxiv.org/pdf/2005.04118.pdf (behavioral testing of the nlp)
2.https://arxiv.org/abs/2111.09963 (behavioral testing of recommender system)
Essentially we will framing the task as NLP task or recommendation task with behavioral metrics satisfied.

eg: keep generating similar sentences with cosine_similarity 0.5 and diversity 0.4 till "n" sentences reached or "x" attempts satisfied.