(enhancement) distros.package_managers.apt.Apt.get_all_packages, use apt_pkg if available instead of subp('apt-cache, pkgnames')

# Bug report
For performance reasons it's less costly to invoke apt_pkg if available instead of shelling out with subp to call the apt-cache command line in [distros.package_management.apt.py](https://github.com/canonical/cloud-init/blob/main/cloudinit/distros/package_management/apt.py#L117).

Now that cloud-init introduced a Recommends dependency per 494bb1ae8cca8cb6f24cb922c9f1a0d90bc117d7, we could leverage the `apt_pkg` module to look at an alternative like the following:

```python
try:
import apt_pkg;
apt_pkg.init();
cache = apt_pkg.Cache();
package_list = [
p.name for p in cache.packages
if p.has_versions
]
except ImportError:
return ... subp(["apt-cache", "pkgnames"]).stdout
```

## Steps to reproduce the problem

## Environment details
- Cloud-init version: 23.3-50-g506e70fa1
- Operating System Distribution: Ubuntu/debian
- Cloud provider, platform or installer type: any

## cloud-init logs