Consider excluding crawlers by default

Anecdotally, we have had a number of users complain about their name and email address being made available to spiders. This could be done by adding a robots.txt file to a pod.

Solid, being a privacy first framework, should consider excluding crawlers by default.

It would be relatively easy subsequently to turn it on via a dashboard, or an app.

The downside to the approach is that an adoption vector of finding people via search engine is lost. And ability to create a social graph.

I'm personally unsure on the trade off here.

Good idea / Bad idea / Thoughts?

ファイル、テスト、エントリーポイントは指定されていません。まず、crawler の除外をデフォルトにすべきかどうか、またユーザーが dashboard や app を通じてオプトインする方法を明確にし、その後、server の pod 配信の動作を調査してください。プライバシーと発見可能性のトレードオフを考慮した、合意済みのポリシーと実装計画が策定されれば完了です。