microsoft / microsoft/mssql-python
Optimization: Can we use chunking for String ( CHAR,VARCAR,NVARCHAR) data type to remove extra memory allocation
@subrata-ms ya está trabajando en esto.
Desde el 26/2/2026.
- Lenguaje dominante
- Python
- Estrellas
- 473
- Forks
- 60
- Merge medio
- 2 d 11 h
- PR fusionados (30 d)
- 36
Descripción
Describe the bug
Currently we allocate fixed size memory ( coulumnsize+1) to handle string data type. This was causing problem for CP1252 character set. We have fix this issue under reported SQLAlchemy bug ( https://github.com/microsoft/mssql-python/issues/435?reload=1?reload=1 ).
However, we do see an opportunity improve the memory allocation for the string data type. One of the consideration/approach could be chunking.
This will help to reduce unnecessary memory allocation considerably during multithreaded execution and for large dataset.
Exception message:
Stack trace:
To reproduce
This is considered as code optimization. Below is the related bug -
https://github.com/microsoft/mssql-python/issues/435?reload=1?reload=1
Expected behavior
All string data type should handle CP1252 character set.
Guía de contribución
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Evaluación
Este issue todavía no se ha evaluado.