koala73 / koala73/worldmonitor
ops(growth): execute the first credentialed SEO and AI visibility measurement cycle
- Dominant language
- TypeScript
- Stars
- 86.6k
- Forks
- 13.1k
- Avg merge
- 8h 4m
- Merged PRs (30d)
- 825
Description
## Context
The implementation from #5928/#5940 and #5944 now exists, but the committed scorecard still reports Search Console, Bing, referral, and Bing AI Performance as unavailable. The manual AI audit covers only 4 of 100 possible query/platform observations. We need the first real measurement cycle before selecting more pages or reports.
This ticket executes the existing measurement contract. It does not add special AI markup, scrape search results, or authorize a new content queue.
## Measurement progress — 6 September 2026
**Status: partial collection; keep this issue open. Hold #5929.** The user opened the World Monitor Search Console property and read-only access worked. Google data is now available, but the new repository baseline and deterministic scorecard have not been generated or committed. The measurements below are a sanitized handoff, not acceptance completion.
### Verified Google measurements
Scope: Web search, all countries and devices, no page filter. Dates use Google's reporting timezone (Pacific Time). Counts were read from exact metric tooltips and supported UI tables; CTR and average position retain the provider's displayed precision.
| Window | Inclusive dates | Google AI impressions | Web search clicks | Web search impressions | Web CTR | Web average position |
| --- | --- | ---: | ---: | ---: | ---: | ---: |
| 28 days | 2026-08-08–2026-09-04 | 49,668 | 112,800 | 339,998 | 33.2% | 5.1 |
| 90 days | 2026-06-07–2026-09-04 | 135,823 | 367,184 | 1,637,619 | 22.4% | 6.0 |
The default “3 months” view covered June 5–September 4 (92 days), so the 90-day window was set explicitly. The 28-day window is contained in the 90-day window; these are not independent before/after periods.
The **Generative AI features** report is accessible. It counts appearances of site links in **AI Overviews and AI Mode**, not visits, conversions, or answers to the registered panel. It exposed Pages, Countries, Devices and Days; filters offered Page, Country and Device. No query dimension or separate Overview/Mode filter was exposed. Google AI impressions are part of Web search reporting: **do not add them to the ordinary Web impression total**. Chart totals aggregate by property, while page rows aggregate by page, so page-row sums need not equal the chart total. [Provider definition and limitations](https://support.google.com/webmasters/answer/16984139).
Selected Google AI page evidence:
| Canonical page | 28-day impressions | 90-day impressions |
| --- | ---: | ---: |
| https://www.worldmonitor.app/ | 42,497 | 118,437 |
| https://www.worldmonitor.app/blog/posts/osint-for-everyone-open-source-intelligence-democratized/ | 2,382 | 4,497 |
| https://www.worldmonitor.app/blog/posts/what-is-worldmonitor-real-time-global-intelligence/ | 1,010 | 3,173 |
| https://www.worldmonitor.app/dashboard | 932 | 6,454 |
| https://www.worldmonitor.app/blog/posts/track-global-conflicts-in-real-time/ | 751 | 1,922 |
| https://www.worldmonitor.app/pro | 723 | 2,301 |
These selected rows are not complete page-family totals or an approved refresh queue. Local sanitized working records also retain 14 selected AI page rows and 15 exact registered target routes for ordinary search in each window. The exposed top 1,000 ordinary-search query rows in each period contained no exact or case-insensitive match for the 25 registered queries. This is **missing evidence, not zero demand**: truncation, omission and privacy suppression are not resolved by that table.
### Indexing and collection limits
- Page indexing was last updated **2026-08-28**: **806 indexed**, **4,154 not indexed**. The 11 exclusion-reason counts reconcile. Redirects and alternate canonicals account for some exclusions; the excluded total is not a defect count. This dated snapshot does not establish September 4 window-end counts or current URL-level index health.
- Native export options are Google Sheets, Excel and CSV. A standard-search CSV action was accepted, but no saved native file was verified through the browser tools. Retained figures are identified as **supported UI transcription**, not a native CSV export. Obtain and verify provider exports on resumption where possible. Google documents that unavailable display values can become zero in downloads; preserve their actual missingness.
- The existing collector has **no field for Google's generative-AI aggregate report**. Preserve this source separately in the sanitized evidence manifest. Importer support needs separate scope based on the actual supported format; do not put these impressions into Bing citation totals or manual observations.
- No credentials, account identifiers, property identifiers, raw restricted exports, personal prompts or user-level data are included in this update.
### Geography decision
**A French VPN is not needed for Search Console, Bing Webmaster or analytics aggregates.** France belongs to the July manual-observation context. The current browser context was UAE; **0 of 125 manual query/surface pairs have been observed** this cycle. Do not classify the unobserved pairs as negative answers.
Before manual collection, choose and document one repeatable context: continue the France/English series with verified location and session conditions, or explicitly start a new UAE/English series and mark July's manual results not directly comparable. The latter was recommended in discussion but has **not been adopted**. Preserve the July evidence and original query digest; do not silently change the issue's comparison requirement. A VPN alone does not reproduce account personalization or the previous session.
### Resume checklist
- [x] Confirm #7766 is in main: #7787 merged on 2026-09-06; verified collection revision was 57060b8c0bec5227879d1901e4382a274507b8ac.
- [x] Establish read-only access to the correct Search Console property and capture the Google totals above.
- [ ] Obtain supported Bing Webmaster/Bing AI Performance and bounded analytics/commerce reports. Google access does not establish access to those sources.
- [ ] Finish supported query-by-page, page-family and current URL-index evidence needed for any page selection; preserve unsupported data as null.
- [ ] Decide and record the manual-audit geography/session protocol, then run the unchanged 25 queries across all five surfaces with actual observation or limitation records.
- [ ] On the next measurement pass, record the latest supported reporting end date and exact 28/90-day windows. Preserve this September 6 snapshot; do not overwrite it with later values.
- [ ] Confirm comparison-copy deployment dates/revisions in the measured period. Recent merges #7625, #7751, #7760, #7761, #7758 and #7788 are candidate confounders; merge dates are not deployment evidence.
- [ ] Generate and validate the new dated baseline, sanitized manifest and deterministic report under the existing ownership boundaries. Reconcile source scope and missingness; retain July as history.
- [ ] Reassess #5929 only after demand, healthy indexation, useful engagement and credible product handoff are supported. No pages are selected yet. Keep #6852's pilot decision with its existing owner.
**No causal ranking, citation, traffic or conversion uplift is claimed.** This update clears the Google-access prerequisite only; it does not complete #6036 or authorize content publication.
## What to do
- Obtain supported read-only exports or access for Google Search Console, Bing Webmaster, Bing AI Performance where available, and bounded analytics/commerce aggregates.
- Run the unchanged 25-query panel across ChatGPT Search, Perplexity, Google AI results where available, and Copilot Search using the documented geography, locale, device, signed-in state, and UTC observation context.
- Populate the 28-day and 90-day search windows, indexation status, page-family rows, referral segments, and product handoff aggregates.
- Generate the next dated baseline and scorecard beside the 2026-07-27 baseline, preserving unavailable and partial values as null with an explicit reason.
- Use the result to select or reject the next two or three page refreshes in #5929.
## Acceptance criteria
- [ ] All 25 queries have one observation per required surface, or an explicit unavailable/platform-limitation record; missing observations are never converted to zero.
- [ ] Google Search Console records indexation, impressions, clicks, CTR, and average position for both trailing windows where access exists.
- [ ] Bing and Bing AI Performance records include cited URLs and grounding queries where access exists, with provider-reported zeroes distinguished from unavailable values.
- [ ] Referral and product outcomes reconcile bounded referrer families and landing-page families to dashboard launches, pricing views, sign-ups, Pro conversions, activations, API actions, and MCP actions where each aggregate exists.
- [ ] The generated scorecard reports coverage by query intent and page family, not only a site-wide score, and compares against 2026-07-27.
- [ ] Search Console/Bing credentials, property identifiers, raw exports, prompt text, and user-level data remain outside the repository.
- [ ] The scorecard records a prioritized experiment queue and explicitly states whether #5929 has enough demand, indexation health, useful engagement, and credible handoff evidence to proceed.
- [ ] No claim of causal traffic, conversion, ranking, or citation uplift is made from this single cycle.
## References
- Measurement contract: docs/research/seo-ai-visibility/README.md
- Query registry: docs/research/seo-ai-visibility/query-set.json
- Current baseline: docs/research/seo-ai-visibility/baselines/2026-07-27.json
- Existing implementation: #5940
- Content decision pages: #5929
## AI search execution handoff — 6 September 2026
## Dependencies and execution readiness
The #7766 implementation dependency is resolved: #7787 merged into main on 2026-09-06. The final five-surface baseline/report still requires the remaining measurement inputs and the documented collection context. Search Console access was established on 2026-09-06; Bing and analytics/commerce access remain unverified. See the dated progress section above. A merged implementation or a complete issue description is not proof of source access or measurement acceptance.
Use the unchanged 25-query panel. The new full target is 125 pairs across ChatGPT Search, Perplexity, Google AI Overview, Google AI Mode and Copilot Search. Preserve unavailable/partial states and the history of the original four-surface baseline. Never reconstruct unobserved July results.
## Ownership
After #7766 releases the measurement contract, one collection owner reserves only new dated files under `docs/research/seo-ai-visibility/baselines/` and `scorecards/`, plus a new dated sanitized evidence manifest beside them. The README, collector, scorecard implementation, query registry and historical observations remain read-only.
If live collection exposes a code defect, report its smallest reproducible case and transfer it to the measurement-code owner through the coordinator. Do not patch those files concurrently or invent an export contract.
## Inputs and prerequisites
Record which authorized Search Console, Bing Webmaster/Bing AI and aggregate analytics exports are available, who supplies them, their reporting windows and scope. Keep raw restricted exports, account identifiers, credentials and user-level data outside the repository. If Search Console exposes a Generative AI report, inspect its actual supported export and record availability; adding importer support is outside this cycle unless separately scoped from the observed format.
For each manual observation, record time, locale, geography, device/session context, exact registered query, platform, visible mention/citation and resolved cited URLs. “No Overview shown” is distinct from “Overview shown without World Monitor.” Use existing missingness fields or the sanitized evidence manifest to explain query-specific limitations. Do not scrape unsupported search interfaces.
## Completion and handoff
- Attempt the complete panel and record actual observations or explicit surface/query limitations. Report observed and unobserved counts; do not treat a missing observation as a negative result.
- Produce and validate a new dated baseline and deterministic report with the updated CLI. Retain the original history and query digest.
- Reconcile supported 28/90-day aggregates with their source scope. Unsupported exports stay unavailable, not estimated.
- Record any comparison-copy deployment during the period as a confounder. Do not claim causal uplift.
- Decide whether #5929 has enough evidence to select specific pages. Provide exact query IDs, canonical URLs, audience, gap, evidence window, proposed next product action and confidence/limitations. If evidence is insufficient, explicitly hold #5929.
- Coordinate `/use-cases/` results with #6852 rather than publishing a second pilot decision.
Implementation defects, missing prerequisites, completed observations and issue acceptance must be reported separately. Preserve the existing #6036 criteria; this addendum does not make unavailable credentialed measurements a pass.
Contributor guide
Research direction
Start with docs/research/seo-ai-visibility/README.md and docs/research/seo-ai-visibility/query-set.json, then compare the 2026-07-27 baseline with the existing implementation and collection inputs. The work is done when the required 28- and 90-day measurements, 25-query observations or limitation records, bounded referral and product aggregates, and a new dated baseline, manifest, and scorecard are generated while preserving nulls and restricted data.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- analytics
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100