aiondemand / aiondemand/aiondemand
[ENH] API validator classes
- Ngôn ngữ chính
- Python
- Star
- 38
- Fork
- 89
- Chỉ số merge pull request
- Không có pull request nào được merge trong 30 ngày
Mô tả
I think it would be good to have API contracts as first-class citizens in order to have an explicit, extensible reference point for unified APIs.
Idea: having a concept of different API contracts explicitply in aiod (first class citizen)
Examples:
* sklearn classifier
* sktime forecaster
* sktime TSclassifier
* pytorch-tabular tabular torch model (not sklearn but one layer below)
Requirements:
* API contract is a first class citizen following strategy pattern
* taggable and retrievable
* contract C can be paired with AI algorithm asset A to check "A satisfies C"
Tentative usage mock design:
```python
from sklearn.ensemble import RandomForestClassifier
from aiod.contracts import SklearnClassifier
SklearnClassifier.istypeof(RandomForestClassifier)
# True
# "RandomForestClassifier is an sklearn classifier"
SklearnRegressor.istypeof(RandomForestClassifier)
# False
SklearnClassifier.runtests(RandomForestClassifier)
# runs tests
# returns test summary or printout
# there are two ways to invoke:
# prior knowledge, i.e., tested somewhere and available as database information (e.g., lookup through batteries included clasess)
# actually re-running the tests, via runtests
```
Methods that all "API contracts" have:
* `istypeof(cls)` - returns `True` or `False`
* `runtests` - runs tests for contract
We can change names of course.
Use cases:
* use case 1 - for automation, e.g.,
* run CRON checks on whether estimator A still satisfies a given contract
* run type and API checks on custom estimators
* this ensures API conformance of estimators offered by the platform, i.e., users can trust if the API contract is "green" they can plug into their workflows
* use case 2: plugin developers for their own estimators
* single endpoint for API contract checking for all AI
* (as long as the API contract is being maintained)
* "API checking service"
Hướng dẫn đóng góp
Đánh giá
Issue này chưa được đánh giá.