MergeInsertBuilder: no API to set WriteParams (allow_external_blob_outside_bases)

Feature Request

MergeInsertBuilder hardcodes WriteParams::default() at src/dataset/write/merge_insert.rs:1594 in the call to write_fragments_internal. There is no public method to override WriteParams or specifically set allow_external_blob_outside_bases.

Impact

Users who store external blob URIs (file://, s3://) via merge_insert get:

External blob URI '...' is outside registered external bases
(dataset root is not allowed). Set allow_external_blob_outside_bases=true to store it as absolute external URI.

The error message tells you to set allow_external_blob_outside_bases=true, but there is no way to do so through MergeInsertBuilder.

Current API

MergeInsertBuilder (lines 393-537) exposes:

• when_matched / when_not_matched / when_not_matched_by_source
• conflict_retries / retry_timeout / commit_retries
• skip_auto_cleanup / use_index / source_dedupe_behavior
• mark_generations_as_merged

None of these allow setting WriteParams or blob-related options.

Suggested Fix

Add one of:

// Option A: expose full WriteParams
pub fn with_write_params(mut self, params: WriteParams) -> Self

// Option B: expose just the blob flag
pub fn with_allow_external_blob_outside_bases(mut self, allow: bool) -> Self

Workaround

For append and overwrite operations, callers can pass custom WriteParams to Dataset::write() or Dataset::append(). Only the merge_insert path is affected.

Version

Lance 4.0.0

Start in src/dataset/write/merge_insert.rs, especially MergeInsertBuilder around lines 393-537 and the write_fragments_internal call at line 1594. Compare the existing builder options with the available WriteParams path and determine how the requested blob setting should be exposed. Done means merge_insert callers can configure external blob URI handling instead of being forced to use WriteParams::default().