CarperAI / CarperAI/Code-Pile

LinusTechTip Programming Forum

Open
#32 1 comment 0 reactions 0 assignees View on GitHub
dataset-request
Dominant language
Python
Stars
110
Forks
32
PR merge metrics
No merged PRs in 30d

Description

## Title

Dataset URL - [LinusTechTip](https://linustechtips.com/forum/20-programming/)

Does the dataset exist in a scraped format? No

## Description
This well-known programming forum, just scanned there have more than 10.000 topics from 2013

## Procedure

- [ ] Attempt to scrape topics from https://linustechtips.com/forum/20-programming/
- [ ] Cleaning, duplicate, formating
- [ ] Convert to lm_dataformat

## Tests

Include a dummy_dataset.parquet file to test your code against. This dummy_dataset should include the columns for the data and metadata associated with the dataset, which will then be converted into the final format for language model consumption, along with an example row or rows that you can verify your code correctly collects. In addition to this file, include the unit test that evaluates your code against this dummy_dataset.

Give an example of the columns and data:

| col1 | col2 | .... |
| ---- | ---- | ---- |
| row1 | row1 | .... |

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.