codefori / codefori/vscode-db2i
Incorrect Precision Loss in DECIMAL/NUMERIC Conversion
- Dominant language
- TypeScript
- Stars
- 76
- Forks
- 54
- Avg merge
- 17m
- Merged PRs (30d)
- 1
Description
### Problem Description
When converting string literals to DECIMAL or NUMERIC types with precision > 16 digits, IBM i DB2 incorrectly rounds the values, losing precision beyond approximately 15-16 significant digits. This behavior occurs even when the target data type explicitly supports higher precision (e.g., DECIMAL(21,0)).
### Expected Behavior
DECIMAL and NUMERIC types with specified precision should preserve exact values up to their declared precision limit (up to 31 digits on IBM i).
### Actual Behavior
Values are rounded/truncated at approximately 15-16 digits during string-to-number conversion, regardless of the target precision specification.
### Reproduction Steps
```sql
-- Test 1: DECIMAL function - INCORRECT ROUNDING
values(decimal('123456789012345678910', 21, 0));
-- Expected: 123456789012345678910
-- Actual: 123456789012345680000 (rounded)
-- Test 2: CAST to DECIMAL - INCORRECT ROUNDING
values(cast('123456789012345678910' as decimal(21, 0)));
-- Expected: 123456789012345678910
-- Actual: 123456789012345680000 (rounded)
-- Test 3: CAST to NUMERIC - INCORRECT ROUNDING
values(cast('123456789012345678910' as numeric(21, 0)));
-- Expected: 123456789012345678910
-- Actual: 123456789012345680000 (rounded)
-- Test 4: Comparison with working case (≤16 digits)
values(cast('1234567890123456' as decimal(16, 0)));
-- Expected: 1234567890123456
-- Actual: 1234567890123456 (correct)
|Context|Version|
|-|-|
|Code for IBM i version|2.18.0|
|Visual Studio Code version|1.105.1|
|Operating System|win32_x64|
Active extensions
```
Account Microsoft (microsoft-authentication): 0.0.1
Autenticazione GitHub (github-authentication): 0.0.2
Azione per server pronto (debug-server-ready): 1.0.0
Browse Lite (browse-lite): 0.3.9
COBOL (cobol): 25.10.27
Code for IBM i Spooled Files (vscode-ibmi-userspooledfiles): 1.2.3
Code for IBM i Walkthroughs (vscode-ibmi-walkthroughs): 0.5.0
Collegamento automatico per debug di Node (debug-auto-launch): 1.0.0
Creazione estensione (extension-editing): 1.0.0
Db2 for IBM i (vscode-db2i): 1.15.1
ESLint (vscode-eslint): 3.0.16
Emmet (emmet): 1.0.0
Error Lens (errorlens): 3.26.0
Esegui merge del conflitto (merge-conflict): 1.0.0
Excel Viewer (gc-excelviewer): 4.2.64
Funzionalità dei linguaggi TypeScript e JavaScript (typescript-language-features): 1.0.0
Funzionalità del linguaggio JSON (json-language-features): 1.0.0
Funzionalità del linguaggio Markdown (markdown-language-features): 1.0.0
GIT (git): 1.0.0
Git Base (git-base): 1.0.0
GitHub (github): 0.0.1
GitHub Copilot (copilot): 1.388.0
GitHub Copilot Chat (copilot-chat): 0.32.5
IBM i Debug (ibmidebug): 3.0.2
IBM i Notebooks (vscode-ibmi-notebooks): 0.0.6
IBM i Project Explorer (vscode-ibmi-projectexplorer): 2.12.3
IBM i Testing (vscode-ibmi-testing): 1.2.3
Java (oracle-java): 24.1.2
Live Server (LiveServer): 5.7.9
Matematica Markdown (markdown-math): 1.0.0
Modifica della configurazione (configuration-editing): 1.0.0
Path Intellisense (path-intellisense): 2.10.0
RPGLE (vscode-rpgle): 0.33.3
Source Orbit (vscode-sourceorbit): 1.4.1
Suggerimento terminale per VS Code (terminal-suggest): 1.0.1
Supporto di npm per VS Code (npm): 1.0.1
TODO Highlight (vscode-todo-highlight): 1.0.5
```
Remote system
|Setting|Value|
|-|-|
|IBM i OS|V7R5M0|
|Tech Refresh|5|
|CCSID Origin|65535|
|Runtime CCSID|1144|
|Default CCSID|1144|
|SSHD CCSID|?|
|cqsh|true|
|SQL|Enabled
|Source dates|Enabled
### Enabled features
|/QOpenSys/pkgs/bin|/usr/bin|/QSYS.lib/ILEDITOR.lib|/QSYS.LIB|/QIBM/ProdData/IBMiDebugService/bin|/QOpenSys/QIBM/ProdData/JavaVM/jdk80|/QOpenSys/QIBM/ProdData/JavaVM/jdk11|/QOpenSys/QIBM/ProdData/JavaVM/jdk17|
|-|-|-|-|-|-|-|-|
|bash|attr|GETNEWLIBL.PGM|QZDFMDB2.PGM|startDebugService.sh|64bit|64bit|64bit|
|chsh|iconv|||||||
|find|setccsid|||||||
|git|uname|||||||
|grep||||||||
|ls||||||||
|md5sum||||||||
|pfgrep||||||||
|sort||||||||
|stat||||||||
|tar||||||||
|tn5250||||||||
Shell env
```bash
BUILDLIB=BFIT1778LB
CURLIB=xxx
EDITOR=vim
HOME=/home/xxx
HOST=
LIBLS=RPGUNIT QGPL QTEMP
LOGIN=bfmi1778
LOGNAME=bfmi1778
MAIL=/var/spool/mail/xxx
OLDPWD=/home/xxx
PASE_USRGRP_LIMITED=N
PATH=/QOpenSys/pkgs/bin:/QOpenSys/usr/bin:/usr/ccs/bin:/QOpenSys/usr/bin/X11:/usr/sbin:.:/usr/bin
PWD=/home/xxx
SHELL=/QOpenSys/pkgs/bin/bash
SHLVL=1
Variants
```json
{
"american": "#@$",
"local": "£§$",
"qsysNameRegex": {}
}
```
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.