ossf / ossf/package-analysis

add urls (http+https) to json report

Open
#585 1 comment 3 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

dynamic analysis enhancement
Dominant language
Go
Stars
912
Forks
74
PR merge metrics
No merged PRs in 30d

Description

It is highly valuable to include URLs audited as part of the analysis in the JSON report. For example (see URLs field):

{
  "Package": {
    "Name": "package-name",
    "Version": "1.2.3",
    "Ecosystem": "npm"
  },
  "CreatedTimestamp": 1674062447,
  "Analysis": {
    "install": {
      "Status": "",
      "Stdout": "...",
      "Stderr": "...",
      "Files": [],
      "Sockets": [],
      "Commands": [],
      "DNS": [],
      "URLs": [
        "http://2oqwmc7o04ssmcdi4e6wn79ys2vnkh.burpcollaborator.net/",
        "https://raw.githubusercontent.com/Gauravbhatia1211/experiment/main/exps.sh"
      ],
    }
  }
}

How to deal with HTTPS traffic?

Had a great chat with @oliverchang, @calebbrown, and @alik-kold. Suggesting adding a MITM-TLS component and routing the TLS traffic through a transparent proxy component able to audit the URLs, body, headers, and more valuable information:

@alik-kold built a working POC with the following architecture (this is the original):

  • adding sslsplit to the analysis container
  • configuring the sandbox container so HTTP + HTTPS traffic will be routed through sslsplit
  • installed the self-signed generated root CA certificate globally on the sandbox container
    2023-01-19 package-analysis create issues - Page 1 (1)

We started working on this feature. let us know WDYT 🙏

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the JSON report shape in the issue and trace how the analysis container records network activity for the install phase. Then review the proposed sandbox-container routing through sslsplit and the HTTPS certificate setup. Done means the report includes audited HTTP and HTTPS URLs and the traffic path works as described.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
networking, security
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.