xai-org / xai-org/x-algorithm

BDSM enforcement policy is undeployable: redacted thresholds [9.99] lack configuration guide

Open
#25 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
33.3k
Forks
5.4k
Avg merge
1d 17h
Merged PRs (30d)
1

Description

Problem

The BDSM (Behavioral Inauthentic-Account Detection) module's safety thresholds
are redacted in the public release to prevent adversarial evasion, but the
documentation and code lack clear guidance on how to create custom enforcement policies.

This undermines the repository's transparency mission, as users cannot understand:

  • What threshold values are expected
  • How to configure their own operating points
  • The relationship between sentinel 9.99 and valid threshold ranges

Current State

Redacted threshold locations:

  • bdsm/runtime/sink_policy.yaml (lines 8-31): All thresholds set to 9.99 sentinel
  • bdsm/runtime/score_results_sink_focal.py (lines 175-207): Default thresholds all set to 9.99

Example of redaction:

thresholds:
  FollowBot:           [9.99, 9.99]
  EngagementAmplifier: [9.99, 9.99]
  RTBot:               [9.99, 9.99]

All probabilities in these fields should be in the range [0, 1], making 9.99
an obvious placeholder (from bdsm/README.md lines 104-112).

Documentation issue:
The README states users should "Supply your own via --policy-file / BDSM_SINK_POLICY"
but provides no template or example showing:

  • The expected structure of each threshold (2-tuple: [tau, lambda])
  • Valid value ranges ([0, 1] for probabilities)
  • Which fields are safe to modify (vs. which control system behavior)

Impact

Users attempting to run BDSM cannot deploy it without either:

  1. Reverse-engineering the policy structure from code parsing
  2. Providing arbitrary 9.99 values that will never trigger enforcement
  3. Guessing appropriate thresholds without domain knowledge

This is particularly problematic for the stated goal of the repo: public transparency
into how the For You feed works
.

Proposed Solution

  1. Create example template: bdsm/runtime/sink_policy.example.yaml

    • Show realistic threshold values in [0, 1]
    • Comment each field explaining its purpose
    • Label which values are likely to be too conservative/aggressive
  2. Update bdsm/README.md:

    • Document the policy file format with concrete examples
    • Explain the 2-tuple structure: [tau, lambda] as threshold and ceiling
    • Add a "Configuration" section showing how to set policy via CLI or env var
    • Link to the example file
  3. Add validation at startup:

    • Warn or fail if thresholds are exactly 9.99 (sentinel value)
    • Log policy version and source on load (already done, but clarify in README)

References

  • README lines 104-112: Explanation of redaction
  • README line 112: "Supply your own via..."
  • sink_policy.yaml: Current redacted file
  • score_results_sink_focal.py lines 175-207: Default policy dataclass
  • score_results_sink_focal.py lines 220-260: Policy loading logic

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with bdsm/README.md lines 104-112, then inspect bdsm/runtime/sink_policy.yaml, score_results_sink_focal.py lines 175-207, and its policy-loading logic at lines 220-260. Done means users have an example policy, documented tuple and value ranges, configuration instructions, and clear handling of the 9.99 sentinel.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, documentation
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.