CorrelAid / CorrelAid/fdb_scraper

Point dcat:contactPoint at a vCard the Civic Data Lab publishes

Open
#4 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
0
Forks
0
PR merge metrics
No merged PRs in 30d

Description

The contact point is currently minted here, as a fragment of the dataset URI:

<https://fdb.cdl.correlaid.org/id/dataset/foerderdatenbank-programme#contact>
    a vcard:Kind, vcard:Organization ;
    vcard:fn "Civic Data Lab"@de ;
    vcard:hasEmail <mailto:mail@civic-data.de> ;
    vcard:hasURL <https://civic-data.de/> ;
.

All three values are restated from civic-data.de, which is the same staleness
argument that #2 settled for the publisher agent: a fact published
authoritatively elsewhere should not have a second version living here. It could
not be settled the same way at the time, because dcat:contactPoint must point
at a vcard:Kind (DCAT-AP 3.0 shapes) and no vCard node existed to point at --
the lab publishes a schema.org node, not a vCard, and Yoast's ContactPoint does
not satisfy the constraint.

Once the lab publishes one, switch to it:

<https://fdb.cdl.correlaid.org/id/dataset/foerderdatenbank-programme>
    dcat:contactPoint <https://civic-data.de/#contactPoint> ;
.

<https://civic-data.de/#contactPoint>
    a vcard:Kind, vcard:Organization ;
    vcard:fn "Civic Data Lab"@de ;
    vcard:hasEmail <mailto:mail@civic-data.de> ;
.

Type, label and the one address stay here because a harvester reads one document
and does not dereference -- same minimum the two agents carry. Everything else
about the contact lives upstream. No explicit pointer to the authority is needed
if the RDF is served at the IRI's own base; if it is served at a different URL
(say the IRI is https://civic-data.de/#contactPoint but the Turtle sits at
/doc/organization.ttl), add rdfs:seeAlso to that document.

What is actually being traded

Pointing at their IRI means this project stops describing a node it owns and
starts asserting triples about one it does not. If the lab changes the address
and this file does not, a consumer merging both graphs holds two
vcard:hasEmail values with nothing to decide between them. That is worse than
a stale copy of our own data: it is a false claim about somebody else's
resource. So the copy has to be kept honest by something other than someone
remembering.

Three ways, in the order they should be tried:

  1. Generate the copy from the upstream document. The build fetches the node
    and copies the triples the shapes require, failing loudly if it is gone or
    incomplete. Staleness is then bounded by the build cycle rather than by
    attention, and it is the same derive-from-the-published-source arrangement
    fdb_scraper.codelists already uses. The pipeline already fetches upstream
    weekly, so the network dependency is not new -- but a decision is needed for
    when civic-data.de is unreachable, and "publish the last known copy, warn"
    is the sane one.
  2. Copy only what cannot drift: a vcard:Kind, vcard:Organization plus
    vcard:fn, no address. The type is what the shapes force and a name changes
    far less often than a mailbox. The cost is a contact point a portal renders
    with no way to make contact, which is most of the point of the property.
  3. Keep the node minted here (the status quo above). Every triple is then
    this project's own claim about its own resource -- "the contact for this
    dataset is the lab, at this address" -- so nothing published here can
    contradict what the lab publishes. The address is still duplicated, but it is
    duplicated as our statement rather than as a copy of theirs.

Do 1; fall back to 3 if the node cannot be served dependably. Either way, copy
nothing beyond what the shapes force: every extra copied triple is one more
thing that can drift.

Blocked on the CDL website publishing the node. The ask there:

  • IRI https://civic-data.de/#contactPoint, or a dedicated document
  • vCard (http://www.w3.org/2006/vcard/ns#), not schema.org
  • Minimum a vcard:Kind, vcard:Organization ; vcard:fn ; vcard:hasEmail ; vcard:hasURL
  • Turtle and/or JSON-LD; a static file plus a rewrite is enough
  • Precedent: https://correlaid.org/#organization, which dct:creator already
    points at, and which is a full worked example rather than just an @id:
    the Turtle is served from /doc/organization.ttl, announced with
    <link rel=alternate type=text/turtle>, and (CorrelAid/website#351) carries a
    #contactPoint vCard built the same way this issue asks the lab for

A dedicated mailbox for the dataset would be worth asking for in the same breath:
with mail@civic-data.de the vCard carries the lab's general address, which is a
copy whichever document it sits in.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the existing fdb_scraper.codelists pattern and the weekly upstream-fetching pipeline; the issue is blocked until civic-data.de publishes the required vCard node. Done means pointing dcat:contactPoint at the published IRI, copying only the required triples with a dependable fallback, or retaining the local node if the upstream source cannot be served.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.