aboutcode-org / aboutcode-org/scancode-toolkit

cocoapods.py generates some JSON-related URLs that lead to a 404

Open
#3,715 3 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
2.6k
Forks
791
Avg merge
1d 12h
Merged PRs (30d)
5

Description

In connection with a `purl2url` [issue](https://github.com/package-url/packageurl-python/issues/143) in `packageurl-python`, I've been exploring the URL-related code in packagedcode's [cocoapods.py](https://github.com/nexB/scancode-toolkit/blob/develop/src/packagedcode/cocoapods.py). With the four PURL spec [examples for cocoapods](https://github.com/package-url/purl-spec/blob/master/PURL-TYPES.rst#cocoapods),

```
pkg:cocoapods/AFNetworking@4.0.1
pkg:cocoapods/MapsIndoors@3.24.0
pkg:cocoapods/ShareKit@2.0#Twitter
pkg:cocoapods/GoogleUtilities@7.5.2#NSData+zlib
```

I got the following results looking for potentially useful JSON files. Using the pattern

```
f'https://raw.githubusercontent.com/CocoaPods/Specs/blob/master/Specs/{hashed_path}/{name}/{version}/{name}.podspec.json'
```

from the `get_urls()` `api_data_url` [variable](https://github.com/nexB/scancode-toolkit/blob/develop/src/packagedcode/cocoapods.py#L462-L467), we get the following URLs, each of which leads to a `404: Not Found` page:

- https://raw.githubusercontent.com/CocoaPods/Specs/blob/master/Specs/a/7/5/AFNetworking/4.0.1/AFNetworking.podspec.json
- https://raw.githubusercontent.com/CocoaPods/Specs/blob/master/Specs/f/c/a/MapsIndoors/3.24.0/MapsIndoors.podspec.json
- https://raw.githubusercontent.com/CocoaPods/Specs/blob/master/Specs/3/b/7/ShareKit/2.0/ShareKit.podspec.json
- https://raw.githubusercontent.com/CocoaPods/Specs/blob/master/Specs/0/8/4/GoogleUtilities/7.5.2/GoogleUtilities.podspec.json

A few lines above that pattern in `get)url()` is a pattern for the `specs_json_cdn_url ` [variable](https://github.com/nexB/scancode-toolkit/blob/develop/src/packagedcode/cocoapods.py#L460)

```
f'https://cdn.cocoapods.org/Specs/{hashed_path}/{name}/{version}/{name}.podspec.json'
```

For the same four cocoapods PURLs, this pattern generates valid URLs to cdn.cocoapods.org JSON files:

- https://cdn.cocoapods.org/Specs/a/7/5/AFNetworking/4.0.1/AFNetworking.podspec.json
- https://cdn.cocoapods.org/Specs/f/c/a/MapsIndoors/3.24.0/MapsIndoors.podspec.json
- https://cdn.cocoapods.org/Specs/3/b/7/ShareKit/2.0/ShareKit.podspec.json
- https://cdn.cocoapods.org/Specs/0/8/4/GoogleUtilities/7.5.2/GoogleUtilities.podspec.json

BTW, some of the URL data in these four `.json` files is not (or no longer) valid, e.g., the "homepage" URL for ShareKit (http://getsharekit.com/) leads to what might be a Turkish-language page -- https://smartem.org/. (The "source"/"git" URL (https://github.com/ShareKit/ShareKit.git) is valid and reflects that the last commit was made in December 2017.)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.