GAP-LAB-CUHK-SZ / GAP-LAB-CUHK-SZ/MVImgNet

Download script issue

Open
#46 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
No language data
Stars
489
Forks
10
PR merge metrics
No merged PRs in 30d

Description

Unable to download the file after n=30
For example:

python download.py --download_folder=MVimages_full --data_name=MVImgNet_origin --skip_first_n=30

terminal output:
LAYER 0: Fetching initial URL: https://cuhko365.sharepoint.com/:f:/s/GAP_Lab_MVImgNet/Eqr0CanUxwhMqCuEqgZ7ZlgBq96EGddlT_WrgLIlZWhyWA?e=p31m4o
LAYER 0: Redirect URL: https://cuhko365.sharepoint.com/sites/GAP_Lab_MVImgNet/Shared%20Documents/Forms/AllItems.aspx?id=%2Fsites%2FGAP_Lab_MVImgNet%2FShared%20Documents%2FMVImgNet_Release%2FMVImgNet_origin&p=true&ga=1
LAYER 0: Query parameters: {'id': '/sites/GAP_Lab_MVImgNet/Shared Documents/MVImgNet_Release/MVImgNet_origin', 'p': 'true', 'ga': '1'}
LAYER 0: Extracted siteAbsoluteUrl: https://cuhko365.sharepoint.com/sites/GAP_Lab_MVImgNet
LAYER 0: Extracted listUrl: /sites/GAP_Lab_MVImgNet/Shared Documents
LAYER 0: Extracted formDigestValue: Present
LAYER 0: Using list_server_relative_url_for_gql: /sites/GAP_Lab_MVImgNet/Shared Documents
LAYER 0: Using current_folder_server_relative_url: /sites/GAP_Lab_MVImgNet/Shared Documents/MVImgNet_Release/MVImgNet_origin
LAYER 0: GraphQL API Endpoint: https://cuhko365.sharepoint.com/sites/GAP_Lab_MVImgNet/_api/v2.1/graphql
LAYER 0: Pagination request URL (first 150): https://cuhko365.sharepoint.com/sites/GAP_Lab_MVImgNet/_api/web/GetListUsingPath(DecodedUrl='/sites/GAP_Lab_MVImgNet/Shared Documents')/RenderListData
LAYER 0: Error in pagination response: {"code": "-1, Microsoft.SharePoint.WebPartPages.DataFormWebPartException", "message": {"lang": "zh-CN", "value": "XmlDefinition \u5c5e\u6027\u5305\u542b\u65e0\u6548 XML\u3002"}}

And then script ends without downloading.
Maybe there are problems with GraphQL page limit?

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with download.py and reproduce the reported command using --skip_first_n=30. Trace the pagination request through the RenderListData response, including the SharePoint XML error shown in the report. Done means the script proceeds past n=30 and downloads the requested files, or reports a clear actionable failure.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli, data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.