Investigate 403 Forbidden errors

## Description

There are a large number of 403 Forbidden errors happening during crawler runs. They appear to be primarily for rubygems coordinates. This appears to be a systemic failure and needs to be investigated.

## Rationale

Systemic failures need to be addressed.

## Definition of Done

- [ ] The cause of the error has been identified.
- [ ] If fixable, handling of the error is fixed
- [ ] if not-fixable, handling of the error is more graceful (_What that means needs to be defined._)

Start by tracing the crawler path that makes requests for rubygems coordinates and records 403 responses; the payload names no files, tests, or entry points. Identify the systemic cause, then verify either corrected error handling or a defined graceful fallback, matching the stated definition of done.

Written by the indexing model from the issue text.