Use Entity-Relationship Model as Definitive Reference
- Dominant language
- No language data
- Stars
- 1.1k
- Forks
- 225
- Avg merge
- 7d 17h
- Merged PRs (30d)
- 3
Description
GTFS is unusual in that the specification describes a file format, and presents itself as a file format, without any direct description of the entity-relationship model that the file format encodes. Most proposed changes seem to involve long discussions about the serialized interchange format of this data model, rather than the data model itself, which is never pictured or formally described.
Would it be beneficial if the definitive point of reference in the GTFS spec, GTFS-RT spec, and all proposed extensions was a formal data model, with the serialization format and documentation following from the model, not the other way around?
Discussions might be much simpler if everyone was just looking at the same entity-relationship diagram. I suspect problems and inconsistencies would stand out more clearly and alternatives would be easier to communicate.
For years it has been a staple of research papers and guides making use of open transit data to reverse-engineer the entity-relationship model from the textual descriptions in the spec. For example:
- https://www.researchgate.net/figure/The-GTFS-Schema-for-the-data-from-JSP-Skopje_fig1_263853949
- https://www.trafiklab.se/docs/using-trafiklab-data/using-gtfs-files/static-gtfs-files/
- https://www.sciencedirect.com/science/article/pii/S2405844020305740
- https://ipeagit.github.io/intro_access_book/4_dados_gtfs.en.html
To me it is really odd that the data model of GTFS is left as an exercise for the reader, which has to be rediscovered and shared over and over (as a topic of scientific research), while it would be the definitive core of many other specifications.
Rather than a purely visual vector drawing, I would expect the data model to be expressed in a formal language, allowing it to be validated and automatically turned into a class diagram, interactively in response to edits. Almost as a convenient side effect, this should allow automatically generating a source code skeleton in any number of languages.
I know some people are cringing at the thought of UML and all the Enterprise Ceremony around it. I am not specifically talking about UML, just anything that gets the job done and is pleasant to work with, and is structured and line-oriented enough for effective diffing and version control.
Here's one of the more promising examples I've identified:
https://github.com/holistics/dbml/
This is used by:
https://dbml.dbdiagram.io
https://dbdocs.io
And here is the full DBML syntax:
https://dbml.dbdiagram.io/docs
Here is a quick sketch data model of a subset of GTFS-static (just an example, probably contains mistakes): https://dbdiagram.io/d/GTFS-Scheduled-Core-6576ceab56d8064ca0c69140
And an image of the live rendered model diagram:
Note how the implicit "service" entity is rendered in the diagram, even though it's not materialized in GTFS feeds. This kind of implicit entities are something that seems to be causing confusion in recent change proposals and having them represented in the spec could make it much easier to communicate.
And the resulting published docs: https://dbdocs.io/abyrd/GTFS-Scheduled-Core?table=stop_times&schema=public&view=table_structure
And the source code used in the above sketch model:
```DBML
Table stops {
stop_id id [pk]
stop_name string [null]
stop_lat float [not null, note: 'range -90 to 90']
stop_lon float [not null, note: 'range -180 to 180']
}
Table routes {
route_id id [pk]
route_name string [not null]
}
Table trips {
trip_id id [pk]
route_id id [ref: > routes.route_id]
service_id string [ref: > services.service_id]
}
Table stop_times {
trip_id id [ref: > trips.trip_id]
stop_sequence integer
stop_id id [ref: > stops.stop_id]
arrival_time hhmmss [note: '''
Conditionally Required:
- Required for first and last stop in a trip
- Required for timepoint=1
- Optional otherwise
''']
departure_time hhmmss
pickup_type pick_drop [null, default: 0]
drop_off_type pick_drop [null, default: 0]
indexes {
(trip_id, stop_sequence) [pk]
}
}
Table services [note: "elided table"] {
service_id id [pk]
}
Table calendar {
service_id id [ref: - services.service_id]
"monday-sunday" bool
start_date integer [note: 'Format YYYYMMDD']
end_date integer [note: 'Format YYYYMMDD']
}
enum pick_drop {
0 [note: "Regularly scheduled"]
1 [note: "No pickup available"]
2 [note: "Phone agency"]
3 [note: "Coordinate with driver"]
}
Table calendar_dates {
service_id id [ref: > services.service_id]
date date
exception_type integer [
note: '''
1 = added
2 = removed
''']
}
```
Contributor guide
Assessment
This issue has not been assessed yet.