aboutcode-org / aboutcode-org/scancode-toolkit

Dockerfile (alpine based) + some quick ideas/suggestions

Abierto
#636 30 comentarios 0 reacciones 0 asignados Ver en GitHub
installation and packaging new feature os support
Lenguaje dominante
Python
Estrellas
2.6k
Forks
791
Merge medio
1 d 12 h
PR fusionados (30 d)
5

Descripción

Hi guys,

Hope you are all well !

I found your great repo while I was working on a personal project aiming to do, also, some code analysis with a distributed bot, working as a virtual agent to build a semi-structured database of meta informations about some open source projects (mainly SQL and graph based).

**Context:**
The goal is to enrich automatically, with a virtual assistant/bot, some of my starred repos by 'left joining', or apply some graphql queries, on some data like, known frameworks detection, [github stats](https://github.com/donnemartin/viz) or [github trends](https://github.com/j-a-c/github-trends), in order to extract more meta data from some of my starred repositories.

In a nutshell, it is all about to have a more convenient local search engines on my starred repos, create a domain specific topic modeling api. By detecting some patterns, I want to manage a dynamic tree of events that could be dedicated to some sub-tasks like to generate Docker/Docker-Compose files from a database a snippets if some dependencies are detected or matched.

**examples of pattern detection to build dynamic dockerfiles:**
- requirements.txt -> append **pyp-pip** to **apk** packages to install in a Alpine based dockerfile, but could do the mapping for an Ubuntu based dockerfile for **apt** package label, **python-pip**.
- Docker alpine: apk --no-cache --no-progress --update pyp-pip
- Docker ubuntu: apt-get install python-pip
- CMakeLists.txt -> add cmake, add some additional helpers for building C/C++ based projects.

## **1. Deploy local instances with Docker/Docker Compose**

For some tasks, it sounds clear that some content lexer, grep, parser would provide faster responses and results; dependencies scanning, licences; **so that s how I found your projects, and that really cool :-)**

So first of all, as a dev ops ^^, I was wondering if it would not be easier to bundle **scancode** into some docker/docker-compose files, of course alpine based in order to keep the size of containers reasonable.

## **2. Some suggestions/features**

So far, I started to give a try to your projects, and it sounds that I will create a fork with some personal ideas/features, close to want the context mentioned above, and wanted to share it with you, so any feedback or experience sharing is tremendously welcomed :-)

**- Administration panel**
- in order to refresh, modernize the server side of scancode with a modern admin ui (reactjs or angularjs)
- https://github.com/aio-libs/aiohttp_admin (Aiohttp + NGAdmin), but you can use some other admin boilerplates from the same authors
- https://github.com/marmelab/admin-on-rest
- https://github.com/marmelab/admin-on-rest-graphql-demo

**- Topic modelling**
- https://github.com/wowdd1/xlinkBook
- https://github.com/leonbrain/FYP_knowledgeGraph
- https://github.com/InformatiCup/InformatiCup2017/blob/master/results/README.md (3 winners projects for github classifiers)
- https://github.com/src-d/blog/blob/master/content/post/github_topic_modeling.md
- docker run srcd/github_topics apache/spark (replace apache/spark with any GitHub repository you wish to analyze).

**- Code parsing/search**
- https://github.com/src-d/enry (aim to be faster than the linguist)
- https://github.com/porter-io/tag-github (for custom mappings and regex rules)
- example of mapping brands, keywords: https://github.com/porter-io/tagg-python/blob/master/tagg/default_defs.json
- https://github.com/google/zoekt (Sounds promising)
- https://github.com/junkblocker/codesearch

**-Starred Github managers**:
- https://github.com/hoop33/limo
- https://voblet.com/github

## **3. Questions related to your roadmap:**
Last question, do you plan to migrate to Python3 your stack of scripts ? Is it something that you have on the roadmap and would like to have a community efforts ?

Ps. sorry for the long post, but I was really inspired by your work ^^, so thanks for reading it all :-)

Please have a great day !

Cheers,
Richard

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.