INCATools / INCATools/semantic-sql
Export formats
- Dominant language
- Python
- Stars
- 69
- Forks
- 7
- Avg merge
- 8m
- Merged PRs (30d)
- 1
Description
As part of KBase's work with ontologies in semsql, it would be very handy to be able to export as DSV directly, rather than going via the sqlite database.
We could put together a file similar to `build.Makefile` to perform the conversions, but I am wary of changes being made in the semsql sqlite build process that we'd have to ensure that we kept up with. I was therefore wondering about adapting the current build command to export either DSV or sqlite, depending on what args it is given.
[build.Makefile](src/semsql/builder/build.Makefile) line 51 onwards:
```
# -- MAIN TARGET --
# A db is constructed from
# (1) triples loaded using rdftab
# (2) A relation-graph TSV
%.db: %.owl %-$(RGSUFFIX).tsv $(PREFIX_CSV_PATH)
rm -f $@.tmp && \
cat $(THIS_DIR)/sql_schema/semsql.sql | sqlite3 $@.tmp && \
echo .exit | sqlite3 -echo $@.tmp -cmd ".mode csv" -cmd ".import $(PREFIX_CSV_PATH) prefix" && \
rdftab $@.tmp < $< && \
sqlite3 $@.tmp -cmd '.separator "\t"' ".import $*-$(RGSUFFIX).tsv entailed_edge" && \
gzip -f $*-$(RGSUFFIX).tsv && \
cat $(THIS_DIR)/indexes/*.sql | sqlite3 $@.tmp && \
echo "ALTER TABLE statements ADD COLUMN graph TEXT;" | sqlite3 $@.tmp && \
(test -d views && find views -maxdepth 1 -name '$(notdir $*)*.sql' -type f -print0 | xargs -0 -I{} sh -c 'sqlite3 $@.tmp< "$$1"' sh {} || echo no views ) && \
mv $@.tmp $@
.PRECIOUS: %.db
```
Alter this so that if the desired output is DSV, the rdftab command is run but the sqlite db loading is omitted. Relevant files, including prefixes, relation graph, and schema SQL could be moved to a directory together or something like that.
Contributor guide
Research direction
Start with src/semsql/builder/build.Makefile, especially the main target around line 51, and trace how rdftab, the relation-graph TSV, prefixes, and sql_schema/semsql.sql feed the SQLite build. Define how the build command selects DSV versus SQLite output, then verify that DSV output skips SQLite loading while the existing database build remains intact.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- sql, sqlite
- Domain
- build-system, databases
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100