musescore / musescore/MuseScore

Speech Synthesis

Open
#17,606 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

feature request needs review
Dominant language
C++
Stars
15.1k
Forks
3.3k
Avg merge
2d 2h
Merged PRs (30d)
91

Description

Your idea

As I do a lot of Pop-Vocal music, and being a tech nerd, I'd love to see a speech synthesis instrument within MuseScore.

Unfortunately when it comes to this topic with regards to notation/DAW-integration, there is just not much out there. Recently I've stumbled accross a new tool developed by a team from University of Barcelona called "cantamus". They have quite an impressive (and not annoying) speech synthesis model especially designed for choir music.
https://cantamus.app/

I really hope, there is room for a collab with that team - they seem quite nice guys - to utilize such a thing as playback engine or plugin inside MuseScore. It would just be a blast and make MuseScore the first notation software that can actually sing.

Problem to be solved

Bring lyrics to life using a speech synthesis engine.

Prior art

I've seen Myriad's approach with their virtual singer engine (which is around for nearly two decades now). Quite impressive, if you think about, that this was released back in 2004. But just sounds a bit out-dated and annoying for nowadays standards.
Check out: https://www.myriad-online.com/en/products/virtualsinger.htm

I think with modern (AI- & non-AI) approaches, we could do much much better nowadays - e.g. Amazon Alexa for example can sing very clean and well for many years now.
https://www.amazon.science/blog/a-simpler-singing-synthesis-system
https://www.amazon.science/publications/singing-synthesis-with-a-little-help-from-my-attention

Additional context

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The issue names no MuseScore files, tests, or entry points; begin by reviewing the proposed Cantamus integration and the playback-engine or plugin approach described. Done would require an agreed implementation scope and a working speech-synthesis instrument that brings lyrics to life, but the issue does not define acceptance criteria.

Written by the indexing model from the issue text.

Assessment

Domain
audio-video-rtc
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.