google-deepmind / google-deepmind/pg19

Request: Add link to community BISAC classification tools

Open
#4 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
No language data
Stars
264
Forks
21
PR merge metrics
No merged PRs in 30d

Description

## Summary

I've created a set of tools that classify the PG-19 corpus using BISAC subject headings and would appreciate a link from the README to help other researchers discover them.

## The Tools

**Repository:** https://github.com/fredzannarbor/pg19-enhancements

### Features

- **BISAC Classification**: Dual-method (keyword + semantic embedding) classification for all 28,044 books into standard publishing categories
- **Gutenberg ID Mapping**: Maps PG-19 internal IDs to canonical Gutenberg numeric IDs for linking to Project Gutenberg
- **Pre-computed Data**: Ready-to-use JSON files with full classification results

### Example Use Cases

- Filter PG-19 by genre (find all fiction, all history, all science, etc.)
- Link PG-19 books directly to Project Gutenberg for full text
- Analyze genre distribution in the corpus
- Create genre-specific subsets for targeted language modeling

### Classification Statistics

| Category | Books | % |
|----------|-------|---|
| FICTION | 8,247 | 29.4% |
| HISTORY | 4,891 | 17.4% |
| BIOGRAPHY | 2,103 | 7.5% |
| RELIGION | 1,847 | 6.6% |
| SCIENCE | 1,523 | 5.4% |

## Suggested README Addition

```markdown
### Community Tools

- [pg19-enhancements](https://github.com/fredzannarbor/pg19-enhancements) - BISAC subject classification and Gutenberg ID mapping for all 28,044 books
```

## Why This Helps

Many researchers using PG-19 need to:
1. Filter by genre/subject for domain-specific experiments
2. Link back to original sources on Project Gutenberg
3. Understand the composition of the corpus

These tools address all three needs without modifying the core dataset.

Thank you for maintaining this valuable benchmark!

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.