Content directory scanning tool

Background

There will be a variety of imperatives for the DAO to review the integrity of assets uploaded to the system over time, including

• checking that the asset hash matches the claimed hash, even if verified by the liason storage provider
• checking that the asset metadata matches the claimed metadat provided by the owner, for example in the context of the content directory
• checking that the asset does not violate some sort of content restriction which is very sensitive, e.g. as indicated by something like PhotoDNA

and probably many other reasons not yet envisioned.

Proposal

Build a tool which allows anyone, and in particular the content directory lead, to do automated downloading of all assets uploaded, and do some sort of client side checking for one or the other type of check (e.g. PhotoDNA seems like an excellent first candidate). A nice feature I think would be very nice would be to have automated alerts sent over email or other transports in order to notify someone of some critical match. This is a very open ended problem description, and it probably should remain so, as it's better to have the tool builder work closely with the content lead to further define exactly how this should work. I would urge starting with the simplest possible thing.

Start by reviewing the proposal and consulting the content directory lead to define the simplest check, since no files, tests, or entry points are named. Clarify the asset download scope, the checking mechanism, and any alerting requirements before implementation. Done should mean that the agreed first check runs over uploaded assets and reports its results.