sassoftware / sassoftware/python-sasctl

Model' statistics not visible in SAS Viya Model Manager Compare section

Open
#202 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
52
Forks
45
Avg merge
23h 38m
Merged PRs (30d)
2

Description

Describe the issue
When using the calculate_model_statistics function, the generated file dmcas_fitstat.json, in the "Data" section of the json, contains float/integer values for any statistics instead of string values.
Importing this file in SAS Model Manager results in "NaN" values when comparing this model to another.
See the screenshot below.
image

Wrapping these values in quoted strings corrects the issue.

Attaching the dmcas_fitstat.json file where the training values where manually wrapped as strings (correctly displayed in SAS Model Manager) and the validate/testing ones are originals from the sasctl function.
dmcas_fitstat.json

To Reproduce
Use the function calculate_model_statistics with any dataset and import that file to a model's files section in SAS Model Manager

Expected behavior
The Compare Section in SAS Model Manager should display the model's statistics values, not NaN.

Version
sasctl==1.10.6

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at calculate_model_statistics and inspect how dmcas_fitstat.json is serialized, then reproduce the issue with a dataset and import the file into SAS Model Manager. Done means the Data section contains values that SAS Model Manager accepts and the Compare section displays statistics instead of NaN.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.