microsoft / microsoft/mssql-python
Optimization: Can we use chunking for String ( CHAR,VARCAR,NVARCHAR) data type to remove extra memory allocation
@subrata-ms arbeitet bereits daran.
Seit 26.2.2026.
- Vorherrschende Sprache
- Python
- Sterne
- 473
- Forks
- 60
- Ø Merge
- 2 T. 11 Std.
- Gemergte PRs (30 T.)
- 36
Beschreibung
Describe the bug
Currently we allocate fixed size memory ( coulumnsize+1) to handle string data type. This was causing problem for CP1252 character set. We have fix this issue under reported SQLAlchemy bug ( https://github.com/microsoft/mssql-python/issues/435?reload=1?reload=1 ).
However, we do see an opportunity improve the memory allocation for the string data type. One of the consideration/approach could be chunking.
This will help to reduce unnecessary memory allocation considerably during multithreaded execution and for large dataset.
Exception message:
Stack trace:
To reproduce
This is considered as code optimization. Below is the related bug -
https://github.com/microsoft/mssql-python/issues/435?reload=1?reload=1
Expected behavior
All string data type should handle CP1252 character set.
Beitragsleitfaden
Erste Schritte
- Lies das ganze Issue und danach den Beitragsleitfaden des Projekts.
- Schreib ins Issue, dass du es übernimmst — das erspart doppelte Arbeit.
- Forke das Repository und arbeite in einem Branch.
- Öffne einen Pull Request, der die Issue-Nummer nennt.
Bewertung
Dieses Issue wurde noch nicht bewertet.