musescore / musescore/MuseScore
Speech Synthesis
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 15.1k
- Forks
- 3.3k
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 91
Description
Your idea
As I do a lot of Pop-Vocal music, and being a tech nerd, I'd love to see a speech synthesis instrument within MuseScore.
Unfortunately when it comes to this topic with regards to notation/DAW-integration, there is just not much out there. Recently I've stumbled accross a new tool developed by a team from University of Barcelona called "cantamus". They have quite an impressive (and not annoying) speech synthesis model especially designed for choir music.
https://cantamus.app/
I really hope, there is room for a collab with that team - they seem quite nice guys - to utilize such a thing as playback engine or plugin inside MuseScore. It would just be a blast and make MuseScore the first notation software that can actually sing.
Problem to be solved
Bring lyrics to life using a speech synthesis engine.
Prior art
I've seen Myriad's approach with their virtual singer engine (which is around for nearly two decades now). Quite impressive, if you think about, that this was released back in 2004. But just sounds a bit out-dated and annoying for nowadays standards.
Check out: https://www.myriad-online.com/en/products/virtualsinger.htm
I think with modern (AI- & non-AI) approaches, we could do much much better nowadays - e.g. Amazon Alexa for example can sing very clean and well for many years now.
https://www.amazon.science/blog/a-simpler-singing-synthesis-system
https://www.amazon.science/publications/singing-synthesis-with-a-little-help-from-my-attention
Additional context
No response
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue names no MuseScore files, tests, or entry points; begin by reviewing the proposed Cantamus integration and the playback-engine or plugin approach described. Done would require an agreed implementation scope and a working speech-synthesis instrument that brings lyrics to life, but the issue does not define acceptance criteria.
Written by the indexing model from the issue text.
Assessment
- Domain
- audio-video-rtc
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100