aboutcode-org / aboutcode-org/scancode-toolkit

Dockerfile (alpine based) + some quick ideas/suggestions

未关闭
#636 30 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
installation and packaging new feature os support
主要语言
Python
星标
2.6k
派生
791
平均合并
1 天 12 小时
30 天内合并 PR
5

描述

Hi guys,

Hope you are all well !

I found your great repo while I was working on a personal project aiming to do, also, some code analysis with a distributed bot, working as a virtual agent to build a semi-structured database of meta informations about some open source projects (mainly SQL and graph based).

**Context:**
The goal is to enrich automatically, with a virtual assistant/bot, some of my starred repos by 'left joining', or apply some graphql queries, on some data like, known frameworks detection, [github stats](https://github.com/donnemartin/viz) or [github trends](https://github.com/j-a-c/github-trends), in order to extract more meta data from some of my starred repositories.

In a nutshell, it is all about to have a more convenient local search engines on my starred repos, create a domain specific topic modeling api. By detecting some patterns, I want to manage a dynamic tree of events that could be dedicated to some sub-tasks like to generate Docker/Docker-Compose files from a database a snippets if some dependencies are detected or matched.

**examples of pattern detection to build dynamic dockerfiles:**
- requirements.txt -> append **pyp-pip** to **apk** packages to install in a Alpine based dockerfile, but could do the mapping for an Ubuntu based dockerfile for **apt** package label, **python-pip**.
- Docker alpine: apk --no-cache --no-progress --update pyp-pip
- Docker ubuntu: apt-get install python-pip
- CMakeLists.txt -> add cmake, add some additional helpers for building C/C++ based projects.

## **1. Deploy local instances with Docker/Docker Compose**

For some tasks, it sounds clear that some content lexer, grep, parser would provide faster responses and results; dependencies scanning, licences; **so that s how I found your projects, and that really cool :-)**

So first of all, as a dev ops ^^, I was wondering if it would not be easier to bundle **scancode** into some docker/docker-compose files, of course alpine based in order to keep the size of containers reasonable.

## **2. Some suggestions/features**

So far, I started to give a try to your projects, and it sounds that I will create a fork with some personal ideas/features, close to want the context mentioned above, and wanted to share it with you, so any feedback or experience sharing is tremendously welcomed :-)

**- Administration panel**
- in order to refresh, modernize the server side of scancode with a modern admin ui (reactjs or angularjs)
- https://github.com/aio-libs/aiohttp_admin (Aiohttp + NGAdmin), but you can use some other admin boilerplates from the same authors
- https://github.com/marmelab/admin-on-rest
- https://github.com/marmelab/admin-on-rest-graphql-demo

**- Topic modelling**
- https://github.com/wowdd1/xlinkBook
- https://github.com/leonbrain/FYP_knowledgeGraph
- https://github.com/InformatiCup/InformatiCup2017/blob/master/results/README.md (3 winners projects for github classifiers)
- https://github.com/src-d/blog/blob/master/content/post/github_topic_modeling.md
- docker run srcd/github_topics apache/spark (replace apache/spark with any GitHub repository you wish to analyze).

**- Code parsing/search**
- https://github.com/src-d/enry (aim to be faster than the linguist)
- https://github.com/porter-io/tag-github (for custom mappings and regex rules)
- example of mapping brands, keywords: https://github.com/porter-io/tagg-python/blob/master/tagg/default_defs.json
- https://github.com/google/zoekt (Sounds promising)
- https://github.com/junkblocker/codesearch

**-Starred Github managers**:
- https://github.com/hoop33/limo
- https://voblet.com/github

## **3. Questions related to your roadmap:**
Last question, do you plan to migrate to Python3 your stack of scripts ? Is it something that you have on the roadmap and would like to have a community efforts ?

Ps. sorry for the long post, but I was really inspired by your work ^^, so thanks for reading it all :-)

Please have a great day !

Cheers,
Richard

贡献指南

打开贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。