audacity / audacity/audacity

Formant preservation in realtime pitch shifting

Open
#5,902 0 comments 0 reactions 0 assignees View on GitHub
Enhancement Request
Dominant language
C++
Stars
18.4k
Forks
2.7k
Avg merge
3d 1h
Merged PRs (30d)
78

Description

### Your idea

Improve quality of non-destructive pitch shifting with formant preservation. This requires algorithmic modifications in the time-and-pitch library.

### Problem to be solved

This is particularly desired for the pitch shifting of voice, which otherwise gets a chipmunk-like quality already at moderate ratios.

Spectrum of an "O":
![image](https://github.com/audacity/audacity/assets/22740106/2763fa04-76d5-4bbe-93a4-cab31410a0d5)

Low-frequency-resolution spectrum of that same "O":
![image](https://github.com/audacity/audacity/assets/22740106/9e51a562-569c-48b1-a4ba-a16439e2cdd8)

Low-frequency-resolution spectrum of that "O" after pitch-shifted one octave up:
![image](https://github.com/audacity/audacity/assets/22740106/a6b1056d-7bf4-4703-9944-39885df406c3)

Frequency resolution is on purpose set low so that harmonics do not stand out individually. Instead, what we see are "spectral envelopes".

Observe the resonances (areas of strong energy) in the original spectrum, e.g. that around 3.6kHz. After pitch shifting, these resonances are also shifted up, and now it is around 7kHz.
Formant preservation aims at keeping these resonances intact, meanwhile still moving the harmonic series alongside the frequency axis.

Note that this could pave the way to offering a more flexible effect, where pitch and timbre can be shifted independently, as does Elastique Pitch:
![image](https://github.com/audacity/audacity/assets/22740106/48219828-7b6a-48e1-a2e2-18ca001afbce)

### Prior art

_No response_

### Additional context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.