sillsdev / sillsdev/silnlp

With paragraph break applied, many second lines in Proverbs begin with punctuation marks

Open
#996 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement pipeline 7: postprocess
Dominant language
Python
Stars
39
Forks
7
Avg merge
1d 9h
Merged PRs (30d)
5

Description

I've been generating a lot of Proverbs drafts for a project and have noted lots of lines that begin with punctuation (comma or period) when I apply paragraph breaks. It happens very frequently with the LBS21 (French) and less frequently with the GNA (Arabic). I've seen this before, but the prevalence here is quite noticeable. Is this something we could rule out in the algorithm?

Proverbs 1 from LBS21 (French)
Image

Prov 3 from GNA (Arabic)
Image

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No source file, test, or entry point is named. Reproduce paragraph-break generation with the LBS21 French and GNA Arabic Proverbs examples, then trace the paragraph-break algorithm to determine where lines acquire leading commas or periods; done means the reported cases no longer produce those leading punctuation marks.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.