provide some level of control to mange the output file sizes when exporting data from synapse using sqlpackage.exe
Open
enhancement
parquet
- Dominant language
- C#
- Stars
- 460
- Forks
- 29
- Avg merge
- 4d 9h
- Merged PRs (30d)
- 7
Description
It would be nice to have some way to control the export particular for larger objects for example consider a 10 billion row partitioned table. buy default this could create a folder per partition.
by the same thinking a smaller table < 1M rows should export to a single file.
Contributor guide
Research direction
Start by examining the export behavior of sqlpackage.exe for Synapse data, focusing on partitioned tables and the number of output files produced. Define how users would control file sizing and how the export should behave for smaller tables, then verify the proposed behavior against both the 10-billion-row partitioned example and the under-1-million-row example.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- sql
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100