Use GenAI to make test scenarios
- Dominant language
- APL
- Stars
- 5
- Forks
- 4
- PR merge metrics
- No merged PRs in 30d
Description
Extremely chaotic but worth considering with how fast the new AI models run.
My idea is to use the OpenAI Package on Tatin(or any other LLM) and make a call to OpenAI(or similar) and ask it to give us some data that it thinks the APL primitive might break over. We should log everything this does.
I think it will give a lot of false positives but non-deterministic testing is a new way of doing things. Its on the same line as chaos testing.
Pros:
- Lots of new data
- Lots of uncommon scenarios that might give aplcores (or errors that are incorrect) because its gonna write something that does not work a lot of times
- Its a new thing ` ¯\(⍣)/¯ `
- (I think) We need more chaos testing
Cons:
- Expensive because it will need a subscription to an AI model (we can use an open source one like LLAMA3, its pretty good)
- Time taking- the time taken for each test will be increased (can be dealt with these only running with the slow tests)
Contributor guide
Assessment
This issue has not been assessed yet.