Flite text to speech doesn't work anymore under headless conditions

Start by verifying whether the SSH-versus-desktop behavior is reproducible in the Linux kernel repository rather than in the Raspberry Pi userland or distribution packages. Reproduce the documented echo "hello world" | flite sequence on the listed Raspberry Pi OS image and compare the two login orders; done would require identifying a kernel-owned cause and a clear regression boundary.

Is this the right place for my bug report?
This repository contains the Linux kernel used on the Raspberry Pi. If you believe that the issue you are seeing is kernel-related, this is the right place. If not, we have other repositories for the GPU firmware at github.com/raspberrypi/firmware and Raspberry Pi userland applications at github.com/raspberrypi/userland. If you have problems with the Raspbian distribution packages, report them in the github.com/RPi-Distro/repo. If you simply have a question, then the Raspberry Pi forums are the best place to ask it.

Describe the bug
Flite text to speech is distorted or no speech when run after ssh login
It has worked okay for years (pulseaudio change is chief suspect). It broke last when pulseaudio introduced in OS late last year. Then it got fixed. Now broken again in latest OS release.

To reproduce
Fresh install latest raspberry pi OS (2021-01-11-raspios-buster-armhf.img).
configure /boot/config.txt as per hifiberry mini-amp zero procedure.
Install flite via apt
Allow ssh
reboot

Then:
ssh pi@raspberry

$> echo "hello world" | flite
Sound may or may not come out - if it does it is very distorted

However:
If you do a GUI Desktop login.
Click on terminal
$> echo "hello world" | flite
The sound usually comes out okay.
If you then ssh into the raspberry (without rebooting) and $> echo "hello world" | flite
It will work okay too.
The sequence is important - it has to be Desktop/Terminal then ssh for it to work

It doesn't work this way round:
If you do ssh then Desktop/Terminal the sound is distorted

Expected behaviour
The following command should produce a recognisable voice.
echo "hello world" | flite

Actual behaviour
Very, very distorted sound or no sound at all.

System
Hardware: Raspberry PI 4 8gb, Hifiberry Zero mini-amp

Raspberry Pi 4 Model B Rev 1.4
PRETTY_NAME="Raspbian GNU/Linux 10 (buster)"
NAME="Raspbian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"

Raspberry Pi reference 2021-01-11
Generated using pi-gen, https://github.com/RPi-Distro/pi-gen, 21090519d85bdaa1615d5d5057d37b09368ea5d2, stage4

Linux droid 5.4.83-v7l+ #1379 SMP Mon Dec 14 13:11:54 GMT 2020 armv7l GNU/Linux
Revision : d03114
Serial : 10000000dd2d719c
Model : Raspberry Pi 4 Model B Rev 1.4
Throttled flag : throttled=0x0
Camera : supported=1 detected=1

Videocore information

Jan 8 2021 14:32:15
Copyright (c) 2012 Broadcom
version 194a85abd768c7334bbadc3f1911c10a7d18ed14 (clean) (release) (start_x)

alloc failures: 0
compactions: 0
legacy block fails: 0

Filesystem information

Filesystem 1K-blocks Used Available Use% Mounted on
/dev/root 14989948 3181368 11147112 23% /
devtmpfs 3874060 0 3874060 0% /dev
tmpfs 4006156 0 4006156 0% /dev/shm
tmpfs 4006156 8756 3997400 1% /run
tmpfs 5120 4 5116 1% /run/lock
tmpfs 4006156 0 4006156 0% /sys/fs/cgroup
/dev/mmcblk0p1 258095 47178 210918 19% /boot
tmpfs 801228 4 801224 1% /run/user/1000

Filename Type Size Used Priority
/var/swap file 102396 0 -2

Package version information

raspberrypi-ui-mods:
Installed: 1.20201210+nmu1
raspberrypi-sys-mods:
Installed: 20210125
openbox:
Installed: 3.6.1-8+rpt5
lxpanel:
Installed: 0.10.0-2+rpt14
pcmanfm:
Installed: 1.3.1-1+rpt25
rpd-plym-splash:
Installed: 0.26

Logs
If applicable, add the relevant output from dmesg or similar.

Additional context
Add any other relevant context for the problem.