HelloZeroNet / HelloZeroNet/ZeroNet
Discussion on Feature request: Document Unit specification
- Dominant language
- JavaScript
- Stars
- 18.8k
- Forks
- 2.3k
- PR merge metrics
- No merged PRs in 30d
Description
# Proposed Feature
Add a way to specify what the document unit (i.e. informations that are in a single page) for each site is. To explain it better I'm going to use ZeroTalk as an example, but first:
## Why do we want such a thing?
* For a possible search engine: to be able to index documents by document unit for each site (this would allow not to scrape html pages, be able to only index actual content rather than fixed header/footer/menu and allow it to be very fast, since it would only need to do some queries to the database)
* For possibly archiving sites: for example after a month that a topic has no more replies (the document unit ids do not change) simply move those "lines" to another site (impossible to do at current state)
## Example of proposed `docunit.json` for ZeroTalk
```
{
"unit_topic": {
"url": "?Topic:",
"id_list": "SELECT topic_id FROM topic",
"db_lines": {
"topic": "SELECT topic_id FROM topic WHERE topic_id = :topic_id",
"comment": "SELECT comment_id FROM comment WHERE topic_uri = :topic_id",
"comment_vote": "SELECT ...",
"topic_vote": "SELECT ..."
}
}
}
```
### Explanation of the above stub
The global object has one key for each "type of document unit" (ZeroTalk has only one kind, i.e. topics, but a size could have many). For each of these we have three compulsory keys:
* `url`: That gives the url (relative to the site one) to get to view that document unit (useful for search engines, but also can be used in reverse: when we are visiting a page and want, as an example, to locally save the content in it for archival purposes, we could parse the url we are in and use the parameters in it to get the important lines in each of the database table).
The parameters are in the form ``: param_type is useful for parsing, and param_name for referring at it, both in the `id_list` and in `db_lines`.
* `id_list`: Database query to select the valid values for parameters (i.e. to get all of the topics the site currently has). The returned table must return under the column `param_name` the possible `param_value`s.
* `db_lines`: This is an object, where the keys are the database tables and the values are the database queries that return the ids identifying the objects pertaining to that document unit.
I'm not so good at explaining things, as you can see by reading the above stuff. I hope it is clear enough. If not, do not hesitate to ask.
#### Possible proposed parameters for url
* ``: `/[0-9]+/`
* ``: `/[a-zA-Z0-9\_\-]+/` (may be changed to account for alphanumeric characters in other languages, diacritics, ...)
* ``: Matches the whole url parameter
## Pro and Cons
**Pro**:
* No need to write code to have it done, just need to agree on a format for it, so *please add your two cents* and propose improvements and other possible use cases for the format.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.