INCATools / INCATools/semantic-sql

Export formats

Open
#96 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
69
Forks
7
Avg merge
8m
Merged PRs (30d)
1

Description

As part of KBase's work with ontologies in semsql, it would be very handy to be able to export as DSV directly, rather than going via the sqlite database.

We could put together a file similar to `build.Makefile` to perform the conversions, but I am wary of changes being made in the semsql sqlite build process that we'd have to ensure that we kept up with. I was therefore wondering about adapting the current build command to export either DSV or sqlite, depending on what args it is given.

[build.Makefile](src/semsql/builder/build.Makefile) line 51 onwards:
```
# -- MAIN TARGET --
# A db is constructed from
# (1) triples loaded using rdftab
# (2) A relation-graph TSV
%.db: %.owl %-$(RGSUFFIX).tsv $(PREFIX_CSV_PATH)
rm -f $@.tmp && \
cat $(THIS_DIR)/sql_schema/semsql.sql | sqlite3 $@.tmp && \
echo .exit | sqlite3 -echo $@.tmp -cmd ".mode csv" -cmd ".import $(PREFIX_CSV_PATH) prefix" && \
rdftab $@.tmp < $< && \
sqlite3 $@.tmp -cmd '.separator "\t"' ".import $*-$(RGSUFFIX).tsv entailed_edge" && \
gzip -f $*-$(RGSUFFIX).tsv && \
cat $(THIS_DIR)/indexes/*.sql | sqlite3 $@.tmp && \
echo "ALTER TABLE statements ADD COLUMN graph TEXT;" | sqlite3 $@.tmp && \
(test -d views && find views -maxdepth 1 -name '$(notdir $*)*.sql' -type f -print0 | xargs -0 -I{} sh -c 'sqlite3 $@.tmp< "$$1"' sh {} || echo no views ) && \
mv $@.tmp $@
.PRECIOUS: %.db
```
Alter this so that if the desired output is DSV, the rdftab command is run but the sqlite db loading is omitted. Relevant files, including prefixes, relation graph, and schema SQL could be moved to a directory together or something like that.

Contributor guide

Open the contributing guide

Research direction

Start with src/semsql/builder/build.Makefile, especially the main target around line 51, and trace how rdftab, the relation-graph TSV, prefixes, and sql_schema/semsql.sql feed the SQLite build. Define how the build command selects DSV versus SQLite output, then verify that DSV output skips SQLite loading while the existing database build remains intact.

Written by the indexing model from the issue text.

Assessment

Tech stack
sql, sqlite
Domain
build-system, databases
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.