Azure-Samples / Azure-Samples/contoso-creative-writer

This AI sample lacks risk & safety evaluation implementation

Open
#308 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Bicep
Stars
428
Forks
1.7k
PR merge metrics
No merged PRs in 30d

Description

As per email sent on 13th Dec with same subject, please double-check that this AI sample implements evaluations. In particular, what we are looking for is:
- [x] Evaluation file(s) (might be a Jupyter notebook, a unit test script, etc.) that evaluates the solution against quality metrics
- [ ] Evaluation file(s) (might be a Jupyter notebook, a unit test script, etc.) that evaluates the solution against at least 2 safety metrics
- [ ] A descriptive section in your readme explaining how evaluation is implemented into the sample.

Contributor guide

Open the contributing guide

Research direction

Inspect the sample's existing evaluation file(s) and README first; the issue does not identify their paths. Confirm how quality evaluation is currently implemented, then add coverage for at least two safety metrics and document the evaluation approach in the README. Done means both unchecked checklist items are addressed.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure
Domain
ai, documentation, testing
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.