xiph / xiph/opus

Should we do `dred_compute_latents` in DTX?

Open
#406 9 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C
Stars
3.3k
Forks
808
PR merge metrics
No merged PRs in 30d

Description

We found that the average cost time doubled in silence segment compared with in voiced segment when enable DRED.

After checking the code, dred_compute_latents will always on even in DTX opus would send empty packet, and minor input will cause a huge increase in GRU's calculations.
To resolve this problem, we tried to enable OPUS_FLOAT_APPROX, but it didn't work, and then we add _mm_setcsr(csr | 0x8040) when do encode in our code, it works.

To significantly save our CPU, We want to ask you experts should we still do dred_compute_latents in DTX? To be specific, could we change to:

#ifdef ENABLE_DRED
    opus_int32 in_dtx = 0;
    if ( st->dred_duration > 0 && st->dred_encoder.loaded && opus_encoder_ctl(st, OPUS_GET_IN_DTX_REQUEST(in_dtx)) == OPUS_OK && !in_dtx ) {
        int frame_size_400Hz;
        /* DRED Encoder */
        dred_compute_latents( &st->dred_encoder, &pcm_buf[total_buffer*st->channels], frame_size, total_buffer, st->arch );
        frame_size_400Hz = frame_size*400/st->Fs;
        OPUS_MOVE(&st->activity_mem[frame_size_400Hz], st->activity_mem, 4*DRED_MAX_FRAMES-frame_size_400Hz);
        for (i=0;i<frame_size_400Hz;i++)
           st->activity_mem[i] = activity;
    } else {
        st->dred_encoder.latents_buffer_fill = 0;
        OPUS_CLEAR(st->activity_mem, DRED_MAX_FRAMES);
    }
#endif

???

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the DRED encoder path around dred_compute_latents and the shown OPUS_GET_IN_DTX_REQUEST check. Reproduce the silence-versus-voiced CPU difference with DRED enabled, then verify DTX packet handling and latent/activity buffer behavior. Done means the project has a reviewed decision and tests or measurements covering the resulting DTX behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
c
Domain
audio-video-rtc, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.