Harmonize Readers' User-Agent
- Dominant language
- HTML
- Stars
- 93
- Forks
- 14
- PR merge metrics
- No merged PRs in 30d
Description
Looking for per-reader downloads stats, we realized (back in 2017! https://github.com/kiwix/container-images/issues/82) that our stats analytical tool (matomo) doesn't identify most of our readers.
While this is a matomo/operations issue/change, it highlighted the fact that our downloader's User-Agents are mising or poorly chosen.
Here's the situation. The identification column was built with [matomo's lib for this](https://github.com/matomo-org/device-detector).
| Reader | User-Agent | Sample | Identified as |
| --- | --- | --- | --- |
| kiwix-desktop | `aria2/{aria-version}` | `aria2/1.36.0` | Client `Type=library, Name=Aria2, Version=1.36.0` |
| kiwix-android | `kiwix-android-version:{VersionCode}` | `kiwix-android-version:231101`, `kiwix-android-version:-1` | OS `Name=Android, ShortName=AND, Platform=, Family=Android, Version=` |
| kiwix macOS | `Kiwix/{ProjectVersion} CFNetwork/{CFNetworkVersion Darwin/{DarwinVersion}` | `Kiwix/173 CFNetwork/1568.100.1.1.1 Darwin/24.0.0` | OS `Name=iOS, ShortName=IOS, Platform=, Family=iOS, Version=18.0` |
| kiwix iOS | `Kiwix/{ProjectVersion} CFNetwork/{CFNetworkVersion Darwin/{DarwinVersion}` | `Kiwix/173 CFNetwork/1568.100.1.2.1 Darwin/24.0.0` | OS `Name=iOS, ShortName=IOS, Platform=, Family=iOS, Version=18.0` |
| Kiwix JS Electron | `xxx KiwixJSElectron/{nwVersion}-E xxx` (used in UA built by Electron) | `Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) KiwixJSElectron/3.4.1-E Chrome/122.0.6261.156 Electron/29.3.1 Safari/537.36` | Client & OS `Type=mobile app, Name=KiwixJSElectron, Version=3.4.1, Name=Windows, ShortName=WIN, Platform=x64, Family=Windows, Version=10` |
- Kiwix JS extensions and PWA uses the browser so there's no specific U-A
- It looks like Kiwix JS Election is the only one that's correct and I believe it's not specifically set but built by Electron.
- Apple version for macOS and iOS are almost identical. matomo is even detecting iOS for the macOS version.
- [User-Agent](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/User-Agent) convention is pretty clear
- kiwix-desktop should set `--user-agent` when configuring aria2
- Apple should manually set the `User-Agent` header of the `URLRequest`
- Build versions as used in Android and Apple are of little help for our use cases. We want human version numbers. U-A allows us to specify both in the same string though.
I suggest we use the following: `Kiwix-{flavor}/{humanVersion} ({platform}/{platformVersion})`. I believe the build number is not useful but this is debatable and can be added as a comment (after the parenthesis).
Which would translate as follows:
```
Kiwix-desktop/2.3.1-4 (Windows/11)
Kiwix-android/3.11.1 (droid/12)
Kiwix-ios/3.6.0 (iOS/18.0)
Kiwix-macos/3.6.0 (macOS/15.0)
```
The most important question being: do we want to consider all readers as `Kiwix` product or should each be its own product? If not, then matomo for instance would group them all under the `Kiwix` product and we'd only be able to distingish readers-originated downloads from others but not compare readers together
```
Kiwix/2.3.1-4 (Windows/11)
Kiwix/3.11.1 (droid/12)
Kiwix/3.6.0 (iOS/18.0)
Kiwix/3.6.0 (macOS/15.0)
```
Once this is settled, we can both patch our matomo image and make a PR for [matomo's repo](https://github.com/matomo-org/device-detector/blob/master/regexes/client/libraries.yml)
```diff
628a629,653
>
> - regex: 'Kiwix-desktop/(\d+[\.\d]+)'
> name: 'Kiwix Desktop'
> version: '$1'
> url: 'https://github.com/kiwix/Kiwix-desktop'
>
> - regex: 'Kiwix-android/(\d+[\.\d]+)'
> name: 'Kiwix Android'
> version: '$1'
> url: 'https://github.com/kiwix/kiwix-android'
>
> - regex: 'Kiwix-ios/(\d+[\.\d]+)'
> name: 'Kiwix iOS'
> version: '$1'
> url: 'https://github.com/kiwix/kiwix-apple'
>
> - regex: 'Kiwix-ipados/(\d+[\.\d]+)'
> name: 'Kiwix iPadOS'
> version: '$1'
> url: 'https://github.com/kiwix/kiwix-apple'
>
> - regex: 'Kiwix-macos/(\d+[\.\d]+)'
> name: 'Kiwix macOS'
> version: '$1'
> url: 'https://github.com/kiwix/kiwix-apple'
```
---
There's also the question of CustomApps. In the logs, I've seen `AndroidDownloadManager` (when deferred to the system?), `QtWebEngine` (kiwix-desktop ??), `WikivoyagebyKiwix` (another branding hell).
Contributor guide
Research direction
Start by reviewing the proposed User-Agent formats and the Matomo device-detector entry at regexes/client/libraries.yml. Trace the reader-specific integration points mentioned for aria2 configuration, Apple URLRequest headers, and Electron-generated headers; done means the product naming decision is settled and the affected readers emit consistent, human-readable User-Agents with matching Matomo detection.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- android, electron, ios, macos
- Domain
- analytics, backend-api-design
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100