sassoftware / sassoftware/sas-copilot

Hallucinate by inventing input variables when generating a random forest model

Open
#44 0 comments 0 reactions 1 assignee View on GitHub

@2TomLi is already working on this.

Since Apr 9, 2024.

bug
Dominant language
TypeScript
Stars
12
Forks
1
Avg merge
4m
Merged PRs (30d)
1

Description

Describe the bug
Hallucinate by inventing input variables when generating a random forest model.

  • incorrect assumption on level of target variable

To Reproduce
Steps to reproduce the behavior:

  1. Use the following comment:

/* Build a Random Forest model from the HMEQ data set with target variable GB and print the performance results to an output data set Results */

  1. Click on Generate Code
  2. Results:
proc forest data=mycas.HMEQ outmodel=mycas.forestModel3;
    target GB / level=interval;
    input DCD1M DCP07 DFXWCAN DFXWUK90 DSIJPND DSIUKAS DSIUSWIL DTBD3M SNYDJCM SNYSECM / level=interval;
    savestate rstore=mycas.forestAstore3;
run;

Expected behavior
Input variables should not be invented, rather replaced by general descriptions + level of target variable is incorrectly assumed.

Screenshots
image

Desktop (please complete the following information):

  • OS: PC
  • Version: beta-wave-2

Reporter's Organization
SAS

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.