canonical / canonical/ubuntu-com-security-api

API response not in plain text format

Open
#135 0 comments 3 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
22
Forks
9
Avg merge
1d 3h
Merged PRs (30d)
2

Description

## Summary

I'm working on an application that consumes the Canonical security API. The interaction between the application and the API was working flawlessly from various months (a year, maybe) ago until last week when the API started to respond with a binary payload.

The application is made in C++ and uses `libcurl` to perform the HTTP queries. And the API URL from where the content is downloaded is https://ubuntu.com/security/cves.json.

IMPORTANT: The binary payload response is not done always, but in a random way. Sometimes the payload has the expected format (JSON) and sometimes it is a binary. Anyway, if I try to download the whole security feed by performing various HTTP queries, it's very likely that one of them will receive the binary payload.

Apparently, the binary payload comes compressed in `br` format. In the image below, you can see the HTTP headers from the server response: At left, when the payload comes in JSON format, and at right, when the payload comes in binary format.
![image](https://github.com/canonical/ubuntu-com-security-api/assets/42682247/a155051b-fcf6-4273-8351-7c22639ede7c)

If I try to decompress it with `brotli`, I get:
```bash
# brotli -d outputfile.br
corrupt input [outputfile.br]
```

Payload example: [outputfile.gz](https://github.com/canonical/ubuntu-com-security-api/files/13440031/outputfile.gz)

```bash
% file outputfile
outputfile: Applesoft BASIC program data, first line number 15
```

## Process

n/a

## Current and expected result

Expected results: Payload always in JSON format.

Current results: Binary payload received in an arbitrary way.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at the https://ubuntu.com/security/cves.json endpoint and inspect the differing response headers and downloaded payloads, including the attached outputfile.gz example. Trace how response compression is handled for repeated requests, then verify that every request returns valid JSON rather than an arbitrary binary payload.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
api, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.