Get URLs from filesystem metadata when importing local files

Most browsers store extra metadata about the files you download so that the OS can tell they are from the internet and shouldn't be trusted as much. This data in most cases includes the URL the file was downloaded from as well as a referrer URL (for example the post page of a booru). It would be nice to have the option of using this data to add known URLs when importing files in the regular file import dialog for example.

Windows

In Windows the data is stored in a NTFS alternate data stream called Zone.Identifier. At its most basic it's an INI file containing the key ZoneId with the Windows URL security zone but many browsers (like Chrome on Windows 10+) also store additional information like HostUrl and ReferrerUrl. For example:

PS E:\Downloads> Get-Content -Path ad8c4fb5bf2af12893eed89bc4026adde28729f6.jpg -Stream Zone.Identifier

[ZoneTransfer]
ZoneId=3
ReferrerUrl=https://safebooru.org/index.php?page=post&s=view&id=3863972
HostUrl=https://safebooru.org//images/3700/ad8c4fb5bf2af12893eed89bc4026adde28729f6.jpg

PS E:\Downloads> Get-Content -Path 14a8805ccf63e6cf7949dcc993e389b7fabf5862214f91ba4c225dac0351fc85.jpg -Stream Zone.Identifier

[ZoneTransfer]
ZoneId=3
ReferrerUrl=https://cdn.discordapp.com/attachments/255819961791021058/942368824152444958/14a8805ccf63e6cf7949dcc993e389b7fabf5862214f91ba4c225dac0351fc85.jpg
HostUrl=https://cdn.discordapp.com/attachments/255819961791021058/942368824152444958/14a8805ccf63e6cf7949dcc993e389b7fabf5862214f91ba4c225dac0351fc85.jpg

There is a Python library to access these alternate data streams: https://github.com/RobinDavid/pyADS

macOS

For macOS it gets a bit more complicated. The metadata is stored in the com.apple.metadata:kMDItemWhereFroms extended attribute.

From the Chromium source code:

// As of Mac OS X 10.4 ("Tiger"), files can be tagged with metadata describing
// various attributes.  Metadata is integrated with the system's Spotlight
// feature and is searchable.  Ordinarily, metadata can only be set by
// Spotlight importers, which requires that the importer own the target file.
// However, there's an attribute intended to describe the origin of a
// file, that can store the source URL and referrer of a downloaded file.
// It's stored as a "com.apple.metadata:kMDItemWhereFroms" extended attribute,
// structured as a binary1-format plist containing a list of sources.  This
// attribute can only be populated by the downloader, not a Spotlight importer.
// Safari on 10.4 and later populates this attribute.
//
// With this metadata set, you can locate downloads by performing a Spotlight
// search for their source or referrer URLs, either from within the Spotlight
// UI or from the command line:
//     mdfind 'kMDItemWhereFroms == "http://releases.mozilla.org/*"'

There is a library called osxmetadata (https://github.com/RhetTbull/osxmetadata) that seems to make accessing this and other macOS metadata pretty easy (you could even get finder tags and import them).

Alternatively, there are a few Python libraries for accessing extended attributes:

• https://pypi.org/project/xattr-compat/
• https://github.com/pts/python-xattr-compat
• https://github.com/iustin/pyxattr
• https://github.com/xattr/xattr

Here's an example I found of accessing kMDItemWhereFroms in Python: https://gist.github.com/dunhamsteve/2889617

Linux

While Linux has extended attributes for this purpose (like user.xdg.origin.url) it has been removed from Chromium and never added to Firefox.

Start by tracing the regular file import dialog and reviewing how the project handles platform-specific file metadata. Compare the Windows Zone.Identifier and macOS kMDItemWhereFroms details in the issue, then define how discovered URLs should be presented or imported; done means the option works safely across the supported platforms.