sassoftware / sassoftware/python-sasctl
Model' statistics not visible in SAS Viya Model Manager Compare section
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 52
- Forks
- 45
- Avg merge
- 23h 38m
- Merged PRs (30d)
- 2
Description
Describe the issue
When using the calculate_model_statistics function, the generated file dmcas_fitstat.json, in the "Data" section of the json, contains float/integer values for any statistics instead of string values.
Importing this file in SAS Model Manager results in "NaN" values when comparing this model to another.
See the screenshot below.
Wrapping these values in quoted strings corrects the issue.
Attaching the dmcas_fitstat.json file where the training values where manually wrapped as strings (correctly displayed in SAS Model Manager) and the validate/testing ones are originals from the sasctl function.
dmcas_fitstat.json
To Reproduce
Use the function calculate_model_statistics with any dataset and import that file to a model's files section in SAS Model Manager
Expected behavior
The Compare Section in SAS Model Manager should display the model's statistics values, not NaN.
Version
sasctl==1.10.6
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at calculate_model_statistics and inspect how dmcas_fitstat.json is serialized, then reproduce the issue with a dataset and import the file into SAS Model Manager. Done means the Data section contains values that SAS Model Manager accepts and the Compare section displays statistics instead of NaN.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api, machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100