bigscience-workshop / bigscience-workshop/data-preparation

Extending this codebase

Open
#39 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
318
Forks
42
PR merge metrics
No merged PRs in 30d

Description

I was looking at this codebase and encountered this bit:
https://github.com/bigscience-workshop/data-preparation/tree/main/sourcing/code_dataset#code-dataset-sourcing

```
The query to create the dataset can be found in query.sql. After creation the dataset was preprocessed with processing.py. Note that there is a bug in the script that filters only for GPL licenses instead of filtering them out. There are instructions to remove the bug but it is left there for reproducibility.
```
This leads me to believe that the code here is meant to be used and investigated "as is" and without modification.
Is this repo primarily meant for reproducibility?

If i wanted to improve and extend it for an independent Dataset building project, should I fork it or work from a branch?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.