OpenEnergyPlatform / OpenEnergyPlatform/oeplatform

The oekg test suite is 45% of the run: where the 400 seconds go

Open
#2,479 0 comments 0 reactions 1 assignee View on GitHub

@jh-RLI is already working on this.

Since Sep 16, 2026.

OEKG-API
Dominant language
Python
Stars
65
Forks
29
Avg merge
15h 25m
Merged PRs (30d)
32

Description

Description of the issue

The oekg app is now 547 of the suite's 1,209 test methods — 45% — and is by far the
most expensive part of a run. Twelve API slices put it there in about six weeks, and slice
twelve (#2477) adds more. This is about keeping the wait short enough not to hold developers
up, before the next map adds another 500.

Measured locally, this branch, against a real Fuseki (python manage.py test oekg):

oekg total 418 s
database setup 12 s
the tests themselves 400 s
tests 547
per test 0.73 s

Database setup is 3% and the per-test fixture is nearly free: setUp makes no HTTP
request, only the cleanup (store.clear) makes one, so the whole fixture is roughly 11 s of
the 400 s. The time is in the test bodies.

Where it goes

cProfile over the whole run (494 s under the profiler, ~23% overhead):

cost cumulative calls each
socket.recv_into 262.8 s (53%) 11,980 22 ms
graph_store._post 267.8 s 5,278 51 ms
— of which update 240.5 s 1,414 170 ms
— of which query 27.3 s 3,864 7 ms
pyshacl.validate 104.6 s (21%) 1,259 83 ms
— of which clone_graph 79.5 s 1,259 63 ms

So roughly 60% waiting on the graph store, and 28% copying RDF graphs that never change.

The graph copying is ours, and it is measurable

rdflib's in-memory store takes 5.38 million add calls across 547 tests. That is not
test data — our own builders account for ~25,000 of them. It is the same static triples
copied over and over:

  • validation.validate_post_state evaluates post_state + label_graph() on every call.
    label_graph() is 2,054 triples and never changes; the merge alone is 25.9 ms
    measured, so ~33 s over 1,259 validations.
  • pyshacl then clones that whole graph again, plus the 1,031-triple shape graph:
    clone_graph is 79.5 s, 76% of all pyshacl time.

Since #2447 a write validates twice (pre-state and post-state), so this doubles per write --
correctly, that is what "judge a write by what it adds" costs, but it doubles the copying too.

Two things ruled out, so nobody re-derives them

  • The storage backend is not the lever. The obvious hypothesis was Fuseki fsyncing per
    update (WF-04 measured exactly that for the old SPARQLUpdateStore path). Measured
    head-to-head on oekg.tests.test_bundle_patch, TDB2 on disk vs FUSEKI_MEM_1=true, twice
    each: 33.0 / 34.3 s on disk, 35.7 / 40.0 s in memory. In-memory is not faster. CI
    already runs the in-memory dataset, so there is nothing to change there either.
  • Logging was not the cost. The debug flood (#2478) made the log unreadable, not the run
    slow.

Ideas of solution

Not decided -- measurements, and one of them is already tried.

  1. pyshacl(..., inplace=True) so the data graph is not cloned. Tried on this branch:
    test_bundle_patch goes 33.0 / 34.3 s → 30.7 s, 54 tests still green, about 10%.
    Cheap, but it is a behaviour change on the validation path -- advanced=True enables
    SHACL rules, which can write into the graph -- so it needs its own review rather than
    riding along in an unrelated pull request. Today it is safe because
    validate_post_state validates a freshly built merge that nothing reads afterwards,
    and that is a property worth pinning by test if the option is taken.
  2. Stop merging the whole label subset into every validation. Only the labels of terms
    the post-state actually references are needed -- a bundle picks a handful out of 2,054.
    This is the larger half of the copying and the one we fully own. Needs care: the shape
    checks rdfs:label datatype and cardinality on picked terms, so the filter has to be
    exactly the referenced IRIs.
  3. Understand the 170 ms update. It is not fsync (ruled out above) and it is not the
    round trip (a query is 7 ms on the same transport). Large INSERT DATA bodies and the
    guarded WHERE are the candidates. Worth one measurement before anyone optimises it,
    because it is 60% of the run and currently unexplained.

Not in scope

Weakening the test seam. The suite runs against a real Fuseki deliberately -- one update
request being one transaction is a property of the engine and the largest measured claim the
OEKG API rests on. An in-memory substitute would pass whatever it was taught. Anything here
has to keep that.

Notes

The profile artifact was taken with
python -m cProfile -o oekg.prof manage.py test oekg --no-input against a local Fuseki; it
is reproducible in a few minutes on any developer machine.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.