CommonCrawl - Consider using fastwarc instead of warcio

Performance speedup of ~25% when tested locally

Here is pseudocode

from fastwarc.warc import ArchiveIterator as FastArchiveIterator
from fastwarc.warc import WarcRecordType
from fastwarc.stream_io import FileStream, GZipStream

# For GZip-compressed local files
def process_warc_new(warc_path: Path | str) -> list[dict]:
    results = []
    filename = warc_path.name if isinstance(warc_path, Path) else warc_path.split("/")[-1]
    archive_iterator = FastArchiveIterator(
        GZipStream(FileStream(warc_path, "rb")), 
        record_types=WarcRecordType.response,
        parse_http=True
    )
    for i, rec in enumerate(archive_iterator):
        try:
            content = rec.reader.read()
            warc_id = rec.record_id[10:-1]
            url = rec.headers['WARC-Target-URI']
            results.append({"url": url, "warc_id": warc_id, "source_id": filename, "content": content})
        except Exception as e:  # noqa: BLE001
            # Handle corruption or other errors
            logger.error(f"Error processing record {i} in {filename}: {e!s}")
            # Try to continue with next record
            continue
    return pd.DataFrame(results)

Start by locating the CommonCrawl processing entry point that currently uses warcio, then compare its record handling with the fastwarc pseudocode in this issue. Check how compressed WARC files, response records, errors, and returned fields are handled. Done means fastwarc is integrated without changing results and the reported performance improvement is confirmed.