alexandre-lecoq / alexandre-lecoq/QueryMultiDb

Prevent massive read of data

Aberta
#40 1 comentário 0 reações 0 responsáveis Ver no GitHub
bug good first issue
Linguagem predominante
C#
Estrelas
2
Forks
5
Métricas de merge de PRs
Nenhum PR com merge em 30d

Descrição

# Reproduction steps #
* Generate an excel file using a query which will read 2000000 rows over several databases.

# Observed #
* It will take hours to read the data, and hours to merge the data. The result might be 9 GB of data. Garbage collection will take hours. It won't be possible to get an excel file in a reasonable time, if at all.

# Expected #
* Prevent doing things that can't work.

# How to fix #
* Hard limit the size of tables at every step (reading, merging, generating). If a table is bigger than the hard limit : drop the table and issue an error log.
* Add 2 parameters --maximumTableRows and --maximumTableColumns to reduce the limit to a certain amount.
* The limit cannot be increased.
* The limit should not apply when results are discarded using --discardresults

The hard limit should be 1048576 rows and 16384 columns per excel specifications.
See https://support.office.com/en-us/article/excel-specifications-and-limits-1672b34d-7043-467e-8e27-269d656771c3

If #48 is implemented those fix would be questionable.

Guia de contribuição

Abrir o guia de contribuição

Avaliação

Esta issue ainda não foi avaliada.

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.