is-crawlable improvements

The intention of the [`is-crawlable`](https://web.dev/is-crawlable/) audit is to help site owners understand whether they are correctly blocking their page from indexing.

However, the audit doesn't handle an edge case in the Google Search [docs](https://developers.google.com/search/docs/advanced/crawling/block-indexing):

> If the page is blocked by a robots.txt file or the crawler can't access the page, the crawler will never see the `noindex` directive, and the page can still appear in search results, for example if other pages link to it.

In this case, it'd be more helpful if the Lighthouse audit flagged that the page _may actually be indexable_ due to their blocking crawlers from discovering the `noindex` directive.

Start with the is-crawlable audit and compare its current behavior with the linked web.dev and Google Search documentation. Done means the audit recognizes when robots.txt or crawler inaccessibility can hide a noindex directive and flags that the page may still be indexable.

Written by the indexing model from the issue text.