xai-org / xai-org/x-algorithm

[Feature Request]: Expose static account-level SimCluster affinities in under-the-hood exports

Open
#210 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
33.3k
Forks
5.4k
Avg merge
1d 17h
Merged PRs (30d)
1

Description

Problem Statement

The under-the-hood export provides visibility into explicit policy penalties (accountLabels, postLabels), but silent reach degradation frequently stems from semantic misclassification rather than punitive enforcement.

If an account’s graph placement drifts into an irrelevant community cluster (e.g., brigading, off-topic viral interactions, or noisy graph neighbors), outbound posts enter the wrong candidate generation pipeline (SimClusters, phoenix). Creators and developers currently have zero diagnostic tools to verify if their account is topicalized correctly.

Proposed Solution

Expose the precomputed macro-level cluster affinities already mapped to the author's user ID within the under-the-hood report.

Because User-to-SimCluster embeddings are precomputed offline in periodic batch jobs, surfacing the top $K$ cluster affinities requires an $O(1)$ read from the feature store rather than real-time impression aggregation.

Proposed Schema Addition
{
  "period": {
    "startDate": "2026-08-01",
    "endDate": "2026-08-31",
    "timezone": "UTC"
  },
  "accountAffinityProfile": {
    "modelVersion": "simclusters-v2",
    "primaryClusters": [
      { "clusterId": 1420, "label": "Software & Open Source", "weight": 0.82 },
      { "clusterId": 804, "label": "AI / Machine Learning", "weight": 0.54 },
      { "clusterId": 31, "label": "General Tech News", "weight": 0.28 }
    ]
  }
}
Why Engineering & Leadership Win Here
  • Kills the "Shadowban" Paranoia with Math: The primary mandate of the under-the-hood export was to eliminate false shadowban accusations. When reach drops, creators inevitably blame silent censorship. Showing their cluster affinities objectively proves that an account isn't suppressed—it is simply rooted in a different or colder graph cluster than the author assumed.
  • First-Principles Compute Efficiency (O(1) Lookup): Unlike real-time impression telemetry, this requires zero streaming pipeline overhead or runtime event logging across billions of feed views. Because user-to-SimCluster embeddings are already precomputed offline in batch feature stores, exporting the top $K$ keys is a trivial key-value fetch during export generation.
  • Bot-Proof & Anti-Gaming: Macro cluster embeddings do not expose per-post ranking coefficients, decay curves, or Heavy Ranker features. It gives creators high-level graph directionality without handing engagement farmers a micro-optimization blueprint.
  • Completes the Open-Source Mandate: Open-sourcing the candidate generation logic in this repository is half the equation; giving developers visibility into their own graph inputs completes the feedback loop.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the under-the-hood export generation and the existing accountLabels and postLabels fields. Then determine whether user-to-SimCluster embeddings and their model version are available to that path; done means the export schema can provide the requested accountAffinityProfile without adding real-time aggregation.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend, data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.