aboutcode-org / aboutcode-org/scancode-toolkit

Dockerfile (alpine based) + some quick ideas/suggestions

Open
#636 30 comments 0 reactions 0 assignees View on GitHub
installation and packaging new feature os support
Dominant language
Python
Stars
2.6k
Forks
791
Avg merge
1d 12h
Merged PRs (30d)
5

Description

Hi guys,

Hope you are all well !

I found your great repo while I was working on a personal project aiming to do, also, some code analysis with a distributed bot, working as a virtual agent to build a semi-structured database of meta informations about some open source projects (mainly SQL and graph based).

**Context:**
The goal is to enrich automatically, with a virtual assistant/bot, some of my starred repos by 'left joining', or apply some graphql queries, on some data like, known frameworks detection, [github stats](https://github.com/donnemartin/viz) or [github trends](https://github.com/j-a-c/github-trends), in order to extract more meta data from some of my starred repositories.

In a nutshell, it is all about to have a more convenient local search engines on my starred repos, create a domain specific topic modeling api. By detecting some patterns, I want to manage a dynamic tree of events that could be dedicated to some sub-tasks like to generate Docker/Docker-Compose files from a database a snippets if some dependencies are detected or matched.

**examples of pattern detection to build dynamic dockerfiles:**
- requirements.txt -> append **pyp-pip** to **apk** packages to install in a Alpine based dockerfile, but could do the mapping for an Ubuntu based dockerfile for **apt** package label, **python-pip**.
- Docker alpine: apk --no-cache --no-progress --update pyp-pip
- Docker ubuntu: apt-get install python-pip
- CMakeLists.txt -> add cmake, add some additional helpers for building C/C++ based projects.

## **1. Deploy local instances with Docker/Docker Compose**

For some tasks, it sounds clear that some content lexer, grep, parser would provide faster responses and results; dependencies scanning, licences; **so that s how I found your projects, and that really cool :-)**

So first of all, as a dev ops ^^, I was wondering if it would not be easier to bundle **scancode** into some docker/docker-compose files, of course alpine based in order to keep the size of containers reasonable.

## **2. Some suggestions/features**

So far, I started to give a try to your projects, and it sounds that I will create a fork with some personal ideas/features, close to want the context mentioned above, and wanted to share it with you, so any feedback or experience sharing is tremendously welcomed :-)

**- Administration panel**
- in order to refresh, modernize the server side of scancode with a modern admin ui (reactjs or angularjs)
- https://github.com/aio-libs/aiohttp_admin (Aiohttp + NGAdmin), but you can use some other admin boilerplates from the same authors
- https://github.com/marmelab/admin-on-rest
- https://github.com/marmelab/admin-on-rest-graphql-demo

**- Topic modelling**
- https://github.com/wowdd1/xlinkBook
- https://github.com/leonbrain/FYP_knowledgeGraph
- https://github.com/InformatiCup/InformatiCup2017/blob/master/results/README.md (3 winners projects for github classifiers)
- https://github.com/src-d/blog/blob/master/content/post/github_topic_modeling.md
- docker run srcd/github_topics apache/spark (replace apache/spark with any GitHub repository you wish to analyze).

**- Code parsing/search**
- https://github.com/src-d/enry (aim to be faster than the linguist)
- https://github.com/porter-io/tag-github (for custom mappings and regex rules)
- example of mapping brands, keywords: https://github.com/porter-io/tagg-python/blob/master/tagg/default_defs.json
- https://github.com/google/zoekt (Sounds promising)
- https://github.com/junkblocker/codesearch

**-Starred Github managers**:
- https://github.com/hoop33/limo
- https://voblet.com/github

## **3. Questions related to your roadmap:**
Last question, do you plan to migrate to Python3 your stack of scripts ? Is it something that you have on the roadmap and would like to have a community efforts ?

Ps. sorry for the long post, but I was really inspired by your work ^^, so thanks for reading it all :-)

Please have a great day !

Cheers,
Richard

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.