tonsky / tonsky/datascript

Querying nested data structures with datascript

Open
#217 7 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Clojure
Stars
5.8k
Forks
318
PR merge metrics
No merged PRs in 30d

Description

I am turning deeply nested trees with datalog via something akin to intension. (I was previously using intension but I've replaced it with some specter, which got me a neat 30% performance boost.)

This means my queries contain stuff like:

[:a ?acct :gizmos _ :gizmo-id ?id]
[:a ?acct :gadgets _ :id ?gadget-id]
[:a ?acct :gadgets _ :color ?color]
[:a ?acct :gizmo-configuration ?cid :gizmo ?id]
[:a ?acct :gizmo-configuration ?cid :gadget ?gadget-id]

It was pretty surprising to see a (working) datascript db input with data terms in the query with more than three entries. When I read the datascript source a while ago, I seem to remember a lot of highly optimized code that expects EAVT/AVET/VEAT.

Is reducing a nested data structure to a bunch of facts the most efficient way to query this kind of data structure? I would prefer to use something declarative with reusable rules like datascript or datamaps over something like specter for most of these queries, although specter is fine for many of them too. As you can see, the queries get repetitive fast, so I need something to describe the behavior.

What are the downsides for not reducing it to a more traditional db of facts? (Presumably there's at least a performance consequence.) Is there a way I can make the fact set faster, even if e.g. it's just using a sorted set?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the Datascript source and comparing the nested query examples with the EAVT/AVET/VEAT assumptions mentioned in the issue. Done would require a maintainer-backed decision about whether this structure is supported efficiently and what concrete change or documentation is needed.

Written by the indexing model from the issue text.

Assessment

Tech stack
clojure
Domain
databases, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.