[BUG]: Datastage converter not giving expected output

Is there an existing issue for this?

• I have searched the existing issues

Category of Bug / Issue

Converter bug

Current Behavior

List of issues where the converter is not working as intended -

SPARKSQL output is not actually clean spark sql - it uses python still
Redundant column lists/mappings with no action being taken
Workflow files outputted as JSON and not yaml
Workflow files outputted with invalid true/false statements in JSON
Stage variables reffering to one another are not working (sub-selects with broken refs)
Looping of stage variables is not possible
Variables are referenced using ${} and not {} - should also account for data types
reference to functions in ndp (which is DataStage references) are kept e.g. ndp.TranformStringToDate
Dataframe in cell x refers to a dataframe/view that was never created. In line 269 a view named L is used, however no such view exists yet (only L_Conv3 which is also the ref in some of the select columns)

Expected Behavior

The output should be sparksql equivalent of databricks

Steps To Reproduce

Will be attaching the files in mail

Relevant log output or Exception details

will be provided separetly

Logs Confirmation

• I ran the command line with --debug
• I have attached the lsp-server.log under USER_HOME/.databricks/labs/remorph-transpilers/<converter_name>/lib/lsp-server.log

Sample Query

Operating System

macOS

Version

latest via Databricks CLI

Start by obtaining the input files and lsp-server.log mentioned in the issue; no reproducible sample or source file is attached here. Separate the listed converter failures into individual cases, then compare each result with the expected Databricks Spark SQL and workflow output. Done requires agreed reproduction cases and fixes covering the confirmed failures.