Allow non-numbers as entity ids
Nobody has claimed this yet.
- Dominant language
- Clojure
- Stars
- 5.8k
- Forks
- 318
- PR merge metrics
- No merged PRs in 30d
Description
This has come up in a couple of issues before, both by developers being against it and for it. To wit: https://github.com/tonsky/datascript/pull/8#issue-32457706 https://github.com/tonsky/datascript/issues/56#issuecomment-73641241
Here's my case for a dynamic entity id type, rather than enforcing number?, as is done in the entid function.
I tried to integrate the high churn real-time data stream that my process receives and updates a in-memory map with. This map has something like the following, pretty common structure:
{ "key1"
{ "attrA" "foo", "attrB" "bar", ... }
"key2"
{ "attrB" "foo", "attrX" "bar", ... }
}
It was easy enough to hook this map up to Datascript (Spossiba Tonsky, you ЯOCK! Datomic in contrast doesn't have published protocols...) by creating a MapToDatascript adapter, just implementing (partially, for the time being) IDB, ISearch, and IIndexAccess, and then query it like so, joining it with a real datascript DB:
(ds/q '[
:find (pull $ ?e [*]) (pull $m ?attr-x [*])
:in $ $m
:where [$ ?e :attr-x ?attr-x]]
@ds-db my-map-adapter)
Except, I had to tweak datascript's entid function - in fact simplifying it - in order to allow string eids, or just any type, for that matter.
Now I can of course create some number out of thin air and attach it to the entity to serve as the entity id, but that is another level of mapping that has to go on on every change of my data, which is thousands per second. What's worse, I now have another id that I need to keep consistent in the face of a machine going down and some failover machine taking over. That complicates things quite a bit.
So why not keep entity ids as flexible - for non-datascript data sources at least - as the datalog queries themselves, that don't care if your patterns are shaped [?e ?a ?v] or something completely different.
That way, integrating datascripts datalog engine with arbitrary data sources will be a lot easier, and depending on the case, more performant.
Is this a route you'd like to go in Datascript? Or is the datascript data model you want other sources to adhere to? In the former case, I'd have a more closed look at the sources and wrap this in a proper pull request, after checking and testing all the ramifications.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the entid function and the partially implemented IDB, ISearch, and IIndexAccess protocols in the MapToDatascript adapter described in the issue. Check the linked pull request and prior issue discussion before deciding whether flexible entity IDs fit the data model; done requires resolving the design and testing all ramifications across supported data sources.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- clojure
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100