Augmented Diff Generation

Since #25 covers a number of different, intertwined pieces, I wanted to break out the augmented diff generation component.

# Overview

Augmented diffs are intended to provide both elements that were actually modified during a time period (the `timestamp` attribute within the `` element should be within the requested window (see **Caveats** below) as well as elements which refer to them. In the case of modifications (indirect or direct) to ways or relations, member (way `nds` or relation `members`) element metadata (coordinates, authorship, etc.) is inlined to avoid the need for additional lookups.

# Output Format(s)

## Row-based

The simplest format that likely meets our needs is row-based, such that it can be combined with the intermediate format containing reconstructed way geometries.

Fields needed are:

* `type`
* `id`
* `geom` (reconstructed in the case of ways and relations)
* `version`
* `minorVersion` (this may not be necessary, as it can be re-calculated using a window function provided suitable `updated` values)
* `updated` (direct or indirect timestamp)
* `changeset` (changeset which directly or indirectly modified the element)
* `uid`
* `user`

## Overpass Augmented OsmChange

For compatibility with [Overpass-generated augmented diffs](https://wiki.openstreetmap.org/wiki/Overpass_API/Augmented_Diffs) (which extend the [OsmChange](https://wiki.openstreetmap.org/wiki/OsmChange) format, as used by [minutely diffs](http://planet.osm.org/replication/minute/), swapping the root `` for ``), OSMesa should match that format.

Overpass augmented diff sequences can be converted to Unix timestamps using ` * 60 + 1347432900` (`2801986` → `2018-01-10T02:41:00.000Z`).

### Example

Simplified version of sequence [2801986](http://overpass-api.de/api/augmented_diff?id=2801986):

```xml

The data included in this document is from www.openstreetmap.org. The data is made available under ODbL.

```

### Element Ordering

Elements should be ordered such that any references to other elements in the same diff appear after the element itself. In practice, this means 1) nodes, 2) ways, 3) relations referencing only nodes + ways, 4) relations referencing relations that previously appeared, 5) ...

Overpass appears to further order by 1) `modify`, 2) `delete`, 3) `create` (although this isn't strictly followed--see the [full augmented diff](http://overpass-api.de/api/augmented_diff?id=2801986)--timestamps may be taken into account.

### Caveats

Overpass aims to answer the question "what changed in the requested time period?" When provided with sequences, they're converted into timestamps (see above) and the minute following is used as the time period being queried.

Due to the way that OSM minutely diffs are created ([more context](https://github.com/drolbr/Overpass-API/issues/346)), they won't contain all changes made during the minute they "represent" if database transactions are open. Rather than retroactively updating elements that changed during a time period, the question we should be asking is "what changes did we find out during the requested time period?" (breaking the assumption of a match between `` `timestamp` values and the time associated with a sequence).

Another way to think about this is "what elements were affected by a given minutely diff" and genuinely just augmenting that diff (populating referenced elements and adding referred-to ones) rather than trying to be clever with time ranges.

In the existing Overpass implementation, `changeset` values for referring elements match the changeset **in which the referring element was modified**. This introduces subtle bugs, e.g. mapbox/osm-adiff-parser#2, when attempting to aggregate changes by changeset (which is another process that requires updating past knowledge because changesets are replicated separately and may remain open (collecting additional changes) for 24 hours after first appearing in the minutely diff replication stream.

## GeoJSON

Bonus points: a GeoJSON FeatureCollection containing 1 or 2 Features (re-assembled geometries) representing the old and new versions of modified elements (indirect or direct).

## JSON

Newline-delimited JSON representation of OsmChange (`action` may be able to be omitted, as it can be inferred by the presence of `` or `version` / `visible` attributes, at least for non-referring elements), for easy consumption?

# Related Work

@geohacker [was previously producing JSON augmented diffs aggregated by changeset](https://www.openstreetmap.org/user/geohacker/diary/40846). The post goes into a bit more detail about edge cases that make augmented diff aggregation difficult.

[osm-replication-streams](https://github.com/mojodna/osm-replication-streams) contains a [parser for Overpass augmented diffs that outputs GeoJSON as described above](https://github.com/mojodna/osm-replication-streams/blob/master/src/parsers/adiff.js).

@kamicut [used osm-replication-streams to build a Lambda function that consumes minutely diffs and outputs them as individual messages on a Kinesis stream](https://github.com/mojodna/osm-pds-pipelines/blob/master/functions/changes/index.js).

Start with issue #25 and compare the requested row-based and Overpass OsmChange formats against the repository's current diff-generation entry point; this issue names no files or tests, so those must be located first. Done means producing augmented diffs with referenced elements, inlined member metadata, the specified ordering, and the documented time-window behavior.

Written by the indexing model from the issue text.