[Bug] autoFetch not fetching all records in subsubquery (i.e. recursive pagination)
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 1.5k
- Forks
- 551
- Avg merge
- 9h 2m
- Merged PRs (30d)
- 3
Description
My hierarchy is:
Event__c -> Blocks__r -> Selected_Blocks__r
When using autoFetch = true and ensuring that maxFetch is far above the number of records returned, I am still seeing done = false and a nextRecordsUrl property for the Selected_Blocks__r.
My code is:
import "dotenv/config";
import type { Connection } from "jsforce";
import { withSalesforceConnection } from "@/utils/salesforce-auth";
const fetchEvent = async (conn: Connection, eventId: string) => {
const query = `
SELECT
Id
, Events_Explore_Points__c
, (
SELECT
Id
FROM
Delegates__r
WHERE
Delegate_Email_Address__c != null
AND Delegate_Status__c = 'Confirmed'
)
, (
SELECT
Id
, Speaker_Bio__c
FROM
Speakers1__r
WHERE
Confirmed_Sessions__c > 0
)
, (
SELECT
Id
, (
SELECT
Id
, Delegate__r.Id
FROM
Selected_Blocks__r
WHERE
Delegate__r.Delegate_Email_Address__c != null
AND Delegate__r.Delegate_Status__c = 'Confirmed'
)
FROM
Blocks__r
)
FROM Event__c
WHERE
Id = '${eventId}'
`;
const results = await conn.query(query, {
maxFetch: 5_000_000,
autoFetch: true,
});
if (!results.records.length) {
throw new Error("No results");
}
return results;
};
const main = async () => {
const eventId = "a00Mn000015cntOIAQ";
const result = await withSalesforceConnection(fetchEvent, eventId);
const { records, ...metadata } = result;
console.log("Event Metadata:", metadata);
console.log("Delegate Size:", records[0].Delegates__r.totalSize);
console.log("Speaker Size:", records[0].Speakers1__r.totalSize);
console.log("Block Records:", records[0].Blocks__r.records);
};
void main().then(() => process.exit(0));
which yields the output:
Event Metadata: { totalSize: 1, done: true }
Delegate Size: 195
Speaker Size: 24
Block Records: [
{
attributes: {
type: 'Block__c',
url: '/services/data/v66.0/sobjects/Block__c/a5DMn000006bHpfMAE'
},
Id: 'a5DMn000006bHpfMAE',
Selected_Blocks__r: {
totalSize: 182,
done: false,
nextRecordsUrl: '/services/data/v66.0/query/0r8xx5WjMNGxMXrA0N-66',
records: [Array]
}
},
{
attributes: {
type: 'Block__c',
url: '/services/data/v66.0/sobjects/Block__c/a5DMn000006bQcnMAE'
},
Id: 'a5DMn000006bQcnMAE',
Selected_Blocks__r: { totalSize: 277, done: true, records: [Array] }
},
{
attributes: {
type: 'Block__c',
url: '/services/data/v66.0/sobjects/Block__c/a5DMn0000078o5FMAQ'
},
Id: 'a5DMn0000078o5FMAQ',
Selected_Blocks__r: { totalSize: 12, done: true, records: [Array] }
}
]
Note that in the first selected block record, we have done: false and a nextRecordsUrl:
Selected_Blocks__r: {
totalSize: 182,
done: false,
nextRecordsUrl: '/services/data/v66.0/query/0r8xx5WjMNGxMXrA0N-66',
records: [Array]
}
I assume the relevant code in jsforce is:
and that this doesn't support recursive nested query pagination.
If so, would it be possible to introduce this as an option?
EDIT:
I've made pull request #1801 and have tested that it works for my query. May need some help with writing test cases.
Note also that this kind of nested queries require minimum API level 58, and has a maximum of 5 nested.
Thank you!
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with src/query.ts around lines 671–686 and 1171–1199, then compare the behavior described in pull request #1801. Done means autoFetch recursively resolves nested Selected_Blocks__r pages so their records are complete and no nested nextRecordsUrl remains; add regression coverage for the shown hierarchy if the repository’s test suite supports it.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- api
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 25/100