aboutcode-org / aboutcode-org/vulnerablecode

bulk_search API returns incomplete / partial data using Chunked Transfer Encoding

Open
#2,120 2 comments 2 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
702
Forks
328
Avg merge
3d 8h
Merged PRs (30d)
3

Description

**Scenario**

Neither with `curl`, nor with `reqests` lib (Python) nor with `OkHttp` (Kotlin) it is possible
to get the complete data of the response, for package: `pkg:maven/com.google.guava/guava@19.0`.
It seems that the server attempts to use Chunked transfer encoding.

It reproduces for both, API v1 + v2.

Note: There are other packages, for which the problem does not happen.

**Reproduce with curl**

```
curl 'https://public.vulnerablecode.io/api/packages/bulk_search' \
-H 'Content-Type: application/json; charset=utf-8' \
--data '{"purls":["pkg:maven/com.google.guava/guava@19.0"]}'
```
gives

`curl: (18) transfer closed with 64255 bytes remaining to read`

**Reproduce with Python**

```
import requests

url = 'https://public.vulnerablecode.io/api/packages/bulk_search'
myobj = {
"purls": [
"pkg:maven/com.google.guava/guava@19.0"
]
}

x = requests.post(url, json = myobj, stream=False)

print(x.text)
```

gives:

`requests.exceptions.ChunkedEncodingError: ('Connection broken: IncompleteRead(97975 bytes read, 64255 more expected)', IncompleteRead(97975 bytes read, 64255 more expected))`

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.