Consider excluding crawlers by default

Anecdotally, we have had a number of users complain about their name and email address being made available to spiders. This could be done by adding a robots.txt file to a pod.

Solid, being a privacy first framework, should consider excluding crawlers by default.

It would be relatively easy subsequently to turn it on via a dashboard, or an app.

The downside to the approach is that an adoption vector of finding people via search engine is lost. And ability to create a social graph.

I'm personally unsure on the trade off here.

Good idea / Bad idea / Thoughts?

未指定任何文件、测试或入口点。首先明确排除 crawler 是否应为默认行为，以及用户如何通过 dashboard 或 app 选择加入，然后检查 server 提供 pod 的行为。当形成一项已达成共识的策略和一份考虑隐私与可发现性之间权衡的实施计划时，即视为完成。