microsoft / microsoft/DacFx

provide some level of control to mange the output file sizes when exporting data from synapse using sqlpackage.exe

Open
#15 0 comments 0 reactions 0 assignees View on GitHub
enhancement parquet
Dominant language
C#
Stars
460
Forks
29
Avg merge
4d 9h
Merged PRs (30d)
7

Description

It would be nice to have some way to control the export particular for larger objects for example consider a 10 billion row partitioned table. buy default this could create a folder per partition.

by the same thinking a smaller table < 1M rows should export to a single file.

Contributor guide

Open the contributing guide

Research direction

Start by examining the export behavior of sqlpackage.exe for Synapse data, focusing on partitioned tables and the number of output files produced. Define how users would control file sizing and how the export should behave for smaller tables, then verify the proposed behavior against both the 10-billion-row partitioned example and the under-1-million-row example.

Written by the indexing model from the issue text.

Assessment

Tech stack
sql
Domain
databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.