Delete redirected pages

From a user

`Links that were once pages and then turn to redirects are our issue. Our content management system auto creates clean URLs. If the title of the page is changed the clean URL is changed and the old URL is redirected to the new URL. The old URL stays in our index unless manually removed. When a link is changed from FETCHED to REDIRECT it would be ideal if the index is removed.`

Start by locating the code that changes a link from FETCHED to REDIRECT and the corresponding index update path. Trace how redirected URLs are handled, then add coverage for removing the old URL from the index and run the relevant crawler tests.

Written by the indexing model from the issue text.