cloudflare / cloudflare/cloudflare-prometheus-exporter
Cloudflare Prometheus Exporter - SQLITE_TOOBIG
- Dominant language
- TypeScript
- Stars
- 181
- Forks
- 39
- Avg merge
- 11h 28m
- Merged PRs (30d)
- 3
Description
This is a fairly Large CF Account (280 domains) API Failures
Running against main branch currently, looks like you have some loosely related issues related to this I am seeing errors attempting to export all the domains
here is a summary.
```
{"msg": "GraphQL error (magic-transit-traffic)", "error": "[GraphQL] account 'XXXXX' does not have access to the path. Refer to this page for more details about access controls: https://developers.cloudflare.com/analytics/graphql-api/errors/",}
{"msg": "Failed to update zone context", "query": "colo-metrics", "error": "string or blob too big: SQLITE_TOOBIG"}
{"msg": "Failed to update zone context", "query": "edge-country-metrics", "error": "string or blob too big: SQLITE_TOOBIG"}
{"msg": "Failed to update zone context", "query": "origin-status-metrics", "error": "string or blob too big: SQLITE_TOOBIG",}
{"msg": "GraphQL error (network-analytics)", "error": "[GraphQL] account 'XXXXX' does not have access to the path. Refer to this page for more details about access controls: https://developers.cloudflare.com/analytics/graphql-api/errors/",}
```
Permissions appear to be correct so I'm assuming access errors are related to rate limiting
I'm not sure of the best way to proceed. I've tried tweaking the polling interval, cache settings, and rate limit, but it hasn't helped much. Perhaps for now I should just exclude these 3 queries?
Please let me know if you have any guidance that may help here.
Contributor guide
Research direction
The report names no files, tests, or entry points. Start by reproducing the export against a large account and inspect the handling of the listed GraphQL queries and zone context updates. Done means identifying the causes of the access and SQLITE_TOOBIG failures and documenting or validating a concrete fix.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- graphql, prometheus, sqlite, typescript
- Domain
- api, databases, observability
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100