Database of Dolthub metadata and data samples

Hi!

I am Ivan, a data engineer obcessed by metadata for a long time. Recently I am started semantic data types identification project. Semantic typing is a method of categorizing data to define how to interpret it. It's common usage in identification of personal information like surnames, names, midnames, birthdays, social identifiers, government identifiers and so on. But many other applications exists too, like automatic documentation for datasets, better data navigation and so on. I wrote a short article about it https://medium.com/@ibegtin/semantic-data-types-systematic-approach-and-types-registry-a2c2a60a467b

There are several products with such feature, but a little number of them are open source and no one yet created an ontology of semantic data types. So I am building registry of semantic data types http://registry.apicrafter.io/ and here is code and raw data https://github.com/apicrafter/metacrafter-registry. Web interface is very simple, all data is available as JSON. Right now there are 162 semantic data types.

And I have reference implementation of several metadata types detection as Metacrafter command line tool https://github.com/apicrafter/metacrafter

One of my goals to is to identify semantic data types and I use open data catalogs like data.gov.uk or data.opendatasoft.com and several other open data portals from Germany, France, Ukraine, Poland, Russia. It require to collect data from these portals and to extract metadata from datasets.

As I understand Dolthub data is well structured and each table field has name, field type and description. Is it possible to publish pr to provide me for research metadata of all public datasets in Dolthub?
Metadata could include:
- package name
- table name
- fieldname
- fieldtype
- field description (if exists)

And, it would be great if it's possible to get samples from public datasets, like 1000 records from each table?

It will be very valuable for my research and I could share it results as text, as new rules and produced datasets on Dolthub.

Best Regards,
Ivan Begtin
email: ivan@begtin.tech