fsspec / fsspec/filesystem_spec
FTP not working with older Windows FTP Servers
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.4k
- Forks
- 490
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 38
Description
First off, thank you for creating this incredible library!
I'm trying to use fsspec to load data from a couple older FTP servers. They don't support the MLSD FTP command and the _mlsd2 functions seems to assume an older UNIX based FTP server.
I suppose for improving it would be to use regular expressions in _mlsd2 to "sniff" for which format it is.
Examples:
ftp.onetgov.net
fsspec URL: "ftp://ftp.onetgov.net/divisions/Infomap/pub/GIS_Downloads/FTP Shapefiles/Address Range.zip"
python FTP dir output:
09-27-20 08:55AM 56006477 Address Points.zip
09-27-20 08:55AM 12246905 Address Range.zip
04-01-20 01:46PM 189059 Adopted_Comp_Plan.zip
09-27-20 08:55AM 70271 Airport Noise Contours.zip
04-01-20 01:46PM 26766 AMA.zip
09-27-20 08:55AM 823689 Benchmarks 88 Datum.zip
09-27-20 08:55AM 6939 Boat Ramps.zip
09-27-20 08:55AM 231407 Brownfield Areas.zip
05-31-12 02:59PM 11940688 CENSUS_BLOCKS_2010.zip
05-31-12 02:59PM 1078784 CENSUS_BLOCK_GROUPS_2010.zip
05-31-12 02:59PM 806391 CENSUS_TRACTS_2010.zip
04-01-20 01:46PM 24693 CIP_Roadways.zip
09-27-20 08:55AM 6547203 Code Enforcement Officer Zones.zip
09-27-20 08:55AM 8551 Colleges and Universities.zip
sdrftp03.dor.state.fl.us
fsspec URL: "ftp://sdrftp03.dor.state.fl.us/Tax Roll Data Files/2020 Final NAL - SDF Files/Citrus 19 Final NAL 2020.zip"
python FTP dir output:
10-05-20 11:43AM 24041148 Brevard 15 Final NAL 2020.zip
10-05-20 11:43AM 1347717 Brevard 15 Final SDF 2020.zip
09-22-20 11:54AM 11041829 Citrus 19 Final NAL 2020.zip
09-22-20 11:55AM 513181 Citrus 19 Final SDF 2020.zip
09-27-20 08:04PM 27358206 Duval 26 Final NAL 2020.zip
09-27-20 08:04PM 1385637 Duval 26 Final SDF 2020.zip
10-01-20 02:45PM 2216030 Gadsden 30 Final NAL 2020.zip
10-01-20 02:45PM 66989 Gadsden 30 Final SDF 2020.zip
10-05-20 05:14PM 1153184 Gilchrist 31 Final NAL 2020.zip
10-05-20 05:14PM 43427 Gilchrist 31 Final SDF 2020.zip
10-02-20 04:43PM 1292859 Gulf 33 Final NAL 2020.zip
10-02-20 04:43PM 66287 Gulf 33 Final SDF 2020.zip
10-05-20 09:39AM 2596017 Hendry 36 Final NAL 2020.zip
10-05-20 09:39AM 128792 Hendry 36 Final SDF 2020.zip
09-30-20 01:20PM 9283336 Hernando 37 Final NAL 2020.zip
09-30-20 01:20PM 534406 Hernando 37 Final SDF 2020.zip
10-01-20 01:54PM 8251812 Indian River 41 Final NAL 2020.zip
10-01-20 01:52PM 390588 Indian River 41 Final SDF 2020.zip
09-29-20 10:25AM 6002346 Monroe 54 Final NAL 2020.zip
09-29-20 10:26AM 225385 Monroe 54 Final SDF 2020.zip
09-23-20 11:09AM 33846865 Orange 58 Final NAL 2020.zip
09-23-20 11:11AM 1556983 Orange 58 Final SDF 2020.zip
10-02-20 10:17AM 11778702 Osceola 59 Final NAL 2020.zip
10-02-20 10:17AM 698317 Osceola 59 Final SDF 2020.zip
10-06-20 01:57PM 41969477 Palm Beach 60 Final NAL 2020.zip
10-06-20 01:57PM 2440714 Palm Beach 60 Final SDF 2020.zip
10-06-20 12:08PM 21268245 Sarasota 68 Final NAL 2020.zip
10-06-20 12:07PM 973079 Sarasota 68 Final SDF 2020.zip
10-01-20 06:58PM 17934070 Seminole 69 Final NAL 2020.zip
10-01-20 06:58PM 948225 Seminole 69 Final SDF 2020.zip
10-02-20 10:31AM 6449364 Walton 76 Final NAL 2020.zip
10-02-20 10:31AM 366020 Walton 76 Final SDF 2020.zip
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the FTP listing logic around the _mlsd2 functions and compare it with the Windows-style dir output shown in the issue. Reproduce the listings against the two example servers if available; done means paths containing these files can be listed and accessed without requiring MLSD.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, networking
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100