forcedotcom / forcedotcom/SFDX-Data-Move-Utility

[ISSUE] - v5: excludeIdsFromCSVFiles=false resolves no external-id lookup (all #N/A); works in 4.36.0

Open
#1,214 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
548
Forks
93
PR merge metrics
No merged PRs in 30d

Description

**Issue summary**

With `excludeIdsFromCSVFiles: false`, SFDMU 5.x resolves **no** external-id lookup — every parent reference in the child CSV comes out as `#N/A`. The same script resolves correctly on 4.36.0.

The failure is silent: SFDMU reports the records as inserted and exits `0` (`Command succeeded`), so the rows reach the org with empty lookups. Nothing in the console output indicates a problem. In our case the resulting nulls were only caught by a post-load record count, after the target org's own validation had rejected every row.

Setting `excludeIdsFromCSVFiles: true` in an otherwise identical script resolves correctly on 5.8.0, so the flag is the trigger rather than the CSV or the object setup.

I could not find `excludeIdsFromCSVFiles` mentioned in the 5.x changelog, so I don't know whether this is a regression or an intended narrowing of the flag's meaning (e.g. "map by Id only"). If it's intended, a note in the docs would help — the current behaviour is indistinguishable from success.

**Steps to reproduce**

Full artifacts, including the `--diagnostic --anonymise` log: https://gist.github.com/ohad-rootstock/6f66b1eb761c3545001f1c5981067895

1. In the target org, create two Accounts named `ProbeParentA` and `ProbeParentB`.

2. `export.json`:

```json
{
"excludeIdsFromCSVFiles": false,
"promptOnIssuesInCSVFiles": false,
"promptOnMissingParentObjects": false,
"objects": [
{ "query": "SELECT Id, Name FROM Account", "externalId": "Name", "operation": "Readonly" },
{ "query": "SELECT LastName, AccountId FROM Contact", "externalId": "LastName", "operation": "Insert", "master": false }
]
}
```

3. `Contact.csv`:

```csv
LastName,Account.Name
ProbeChildOne,ProbeParentA
ProbeChildTwo,ProbeParentB
```

4. Run:

```
sf sfdmu run --sourceusername csvfile --targetusername --path . --noprompt
```

5. Repeat with `sfdmu@4.36.0`, and separately with `"excludeIdsFromCSVFiles": true`.

**Expected behavior**

`Contact.AccountId` populated with the Id of the Account matching on `Name` — which is what 4.36.0 does, and what 5.8.0 does when the flag is `true`.

**Actual behavior**

| SFDMU | `excludeIdsFromCSVFiles: false` | `: true` |
| --- | --- | --- |
| 4.36.0 | AccountId resolved | — |
| 5.8.0 | **AccountId = `#N/A`** | resolved |

The parent records *are* retrieved — the log shows the `Readonly` Account query returning both records from the target — but the mapping is not applied to the child.

Environment: SFDMU 5.8.0 vs 4.36.0, @salesforce/cli 2.147.7 and 2.148.3, Node 24, macOS and Windows (reproduced on both).

**Log file (required)**

`--diagnostic --anonymise` log from the failing 5.8.0 run, in the gist above:
https://gist.github.com/ohad-rootstock/6f66b1eb761c3545001f1c5981067895#file-sfdmu-5-8-0-diagnostic-log

**\_target.csv file (if relevant)**

```csv
"Id","LastName","AccountId","Errors"
"003O3000021bdonIAA","ProbeChildOne","#N/A","#N/A"
"003O3000021bdooIAA","ProbeChildTwo","#N/A","#N/A"
```

Also in the gist as `sfdmu-5.8.0-Contact_insert_target.csv`.

---

**A second, possibly related behaviour change** — happy to split this into its own issue if you'd prefer.

5.x appears to send a **blank CSV cell as `""`** where 4.x omitted the field. Where the target field is a checkbox, that now fails:

```
INVALID_TYPE_ON_FIELD_IN_RECORD: ... value not of required type: :Some_Checkbox__c
```

Under 4.x the blank cell was omitted and the org applied the field's default, so the row inserted. Under 5.x one blank cell in one column cost exactly one row out of 6016 in our load — again with an overall exit code of `0`.

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the issue with export.json, Contact.csv, and the `sf sfdmu run` command, comparing SFDMU 5.8.0 with 4.36.0 and the flag set to true. Inspect the linked diagnostic log and generated target CSV to trace why the parent mapping becomes `#N/A` and how blank cells are emitted. Done means external IDs resolve with the flag false and blank checkbox fields retain the expected behavior without a successful exit masking failures.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
cli, data-engineering
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.