Technical design: content localization and internationalization in 3.x

This ticket is concerned with the localization and internationalization of user-editable content in Apostrophe 3.x sites. For static text, see: https://github.com/apostrophecms/apostrophe/issues/3204

## Module Design

In 3.x, content localization will be a standard feature in a forthcoming minor release update of 3.x core. It will not be a separate module as in 2.x. This is because the feature is significantly simpler in 3.x, already "baked into" the database design, and essentially mandatory for website creation in certain countries, i.e. Canada. Thus it meets the standard of "things you must have to have a website" that we try to apply to core apostrophe features. Of course this does not mean you must configure any locales.

## Configuration

Configuration of the available locales will take place in a `@apostrophecms/i18n` module, supporting a nested format as in 2.x:

```
options: {
locales: [
{
name: 'default',
label: 'Default',
private: true,
children: [
{
name: 'en-gb',
label: 'England'
},
{
name: 'en-us',
label: 'United States'
},
{
name: 'fr-fr',
label: 'France'
}
]
},
],
defaultLocale: 'en-gb'
}
```

More than one locale at the root level is permitted. The hierarchy is mainly for user interface purposes when selecting locales as an editor, however it is also applied to auto-replication as described below.

The `defaultLocale` is that which is assumed if the locale cannot be inferred from the request and it does not have to be a root locale.

## Private locales

Private locales can never be viewed by logged-out users. To simplify what was complex in 2.x permissions, all users with the `contributor` role or better can view private locales.

## Permissions

In the core open source release of Apostrophe, all users with the `contributor` role or better can edit across locales, although they may not be able to publish as appropriate to their role. Per-locale permissions will be considered an enterprise feature. Note that the `contributor` role in 3.x solves most problems that would otherwise arise from this.

## Prefixes and hostnames

Locales will continue to support `prefix` and `hostname` options as in 2.x. The `defaultLocaleByHostname` option will also be supported as in 2.x.

The current locale will not be stored in a server-side session variable. When necessary, i.e. because prefixes and hostnames were not set for development and early content creation, it will be kept in a cookie on the browser side. Otherwise it will be inferred from prefixes and hostnames as in 2.x.

## Database structure

Unlike Apostrophe 2.x, Apostrophe 3.x has begun its life with a significant portion of this feature already implemented. That is because, in 3.x, a distinction between "draft" and "published" content always exists. This is implemented through the use of distinct MongoDB documents that are related to one another by a shared `aposDocId` property.

Also unlike 2.x, in Apostrophe 3.x the `_id` property has a constant `aposDocId` part, followed by parts that identify the "locale" and the "mode." Although 3.x does not yet have a UI for localization, it already sets a fixed "locale," `en`, on new documents in anticipation of this feature.

Thus in 3.x a typical page has the following properties relevant to localization:

```
_id: "xyz:en:draft",
aposDocId: "xyz",
aposLocale: "en:draft",
aposMode: "draft"
```

Note that the `_id` property includes all of the information present in the other three, however here is some denormalization (duplication) for convenience and for efficient queries.

If a document exists in two language locales, `en` and `fr`, then there will be four copies, with the following `_id` values:

```
"xyz:en:draft"
"xyz:en:published"
"xyz:fr:draft"
"xyz:fr:published"
```

## Replication

In A3, there is no guarantee that a document exists in multiple locales, except for the following documents which are always replicated across all locales:

* The home page.
* Other parked pages.
* The `@apostrophecms/global` document, which contains global settings.
* Documents directly related to the above three categories of documents. "Directly related" means that the schema of the document itself or the schema of its own widgets contain a `relationship` field pointing to those documents.
* Auto-replication to a new locale is based on the nearest parent available in the hierarchy, according to the same algorithm used in 2.x when `replicateAcrossLocales: false` is set.

The above approach is also available in 2.x if the `replicateAcrossLocales: false` option is set. In 3.x this is the default and only behavior, to prevent databases from becoming unreasonably large due to mass replication when much of the content is locale specific.

## Exceptions

### Never-published drafts and archived documents

A document that has never been published will exist in the `draft` mode but not the `published` mode. `lastPublishedAt` will be nullish.

A document that is in the archive will also exist in the `draft` mode only, and `lastPublishedAt` will again be nullish.

### localized: false

Piece types with the `localized: false` flag are not localized at all. They have no `aposLocale` or `aposMode`. Their `aposDocId` is identical to their `_id`. This feature is typically reserved for user accounts and other content types for which localization and drafts do not make sense. Page types cannot use `localized: false`. Piece types that require translation (localization) should never use `localized: false`.

### autopublish: true

Piece types with the `autopublish: true` flag *do* have locales and modes, and both `draft` and `published` modes will exist. However, at the user interface level, Apostrophe will guarantee that when such pieces are updated the changes are immediately published. This is useful when localization is appropriate but approvals are not. Images are a good example: approving images is usually a waste of effort because they do not appear on pages unless selected for that purpose.

A strength of `autopublish: true` is that it leaves open the possibility of changing your mind and setting it to false later for that type. This is not currently possible with `localized: false`.

## Operations across locales

In 2.x, operations across locales included:

* "Replicate," i.e. the feature which invites you to create a document in "fr" that already exists in "en" at the time you first try to switch to "fr" as an editor;
* "Export," which attempts to copy only the changes made in the latest commit; and
* "Force Export," which copies the entire document's current draft mode content to another locale's current draft mode.
* "Force Export Related," which also copies "related" documents such as images when copying a document.

In 3.x, this feature set is simplified based on the actual experiences of several enterprise customers:

* "Replicate" will continue to be available.
* The new "Export" feature, now called "Localize" will be replaced with the behavior of the old "Force Export" feature. That is, when a user "exports" from locale A to locale B, Apostrophe will always copy the entire document's content to locale B.
* The old "Export" behavior will be removed.
* "Force Export Related" would continue to be available, under a new name, as part of "Localize." However based on user experiences it will definitely always ask which related types you want to localize, and will probably have a provision to always exclude certain types. For instance, if your page has an explicit relationship to the home page, you still don't want the home page exported also every time you export that page. Pages in general will always be excluded.

Regarding the removal of the old "Export " feature, this change is being made because it was only possible to use it well if the document's changes always began in a "main" locale and were then exported gradually to child locales. Any deviation from this pattern makes it impossible for Apostrophe to detect the relevance of the changes to the other locale, and so they were not exported. End users eventually just used the "Force Export" button. So we are taking that simpler approach only in 3.x.

However, 3.x does have a forthcoming "copy and paste" feature for widgets, which should be a very practical alternative when only changes to one widget need to be copied to another locale.

## REST API

The localize operation will be available via a distinct REST API endpoint, i.e. `POST /:_id/localize`. While it can technically be reproduced using only the existing POST and PATCH end points, it is better to have operations which provide a way to track the origin of a document, i.e. which locale it was most recently localized from.

## Relationships (joins) across locales

Relationships (formerly called joins in 2.x) had limitations in 2.x. It was not possible to have a relationship from a type exempt from workflow to a type included in workflow. This is because every locale of each document had a distinct, basically random `_id`.

In 3.x, relationships are always resolved based on the `aposDocId`, not the entire `_id`. As a result, there is no problem when resolving relationships. Piece types marked `localized: false` may have relationships to others, and vice versa.