Modified version of jcode saves LOTS on tokens!!

Current binary: jcode v0.11.0 (eefff4c5-interlang-perturn-sidebar)

Executive summary

I integrated an llm-interlang-inspired symbolic compression layer into Jcode's outbound model request path. The goal is to reduce repeated input context sent to the external model while preserving exact semantics. The implementation currently uses a verified, reversible protocol based around self-describing <il:v1> blocks plus a stronger protocol reminder in IL verified mode.

The implementation is live in Jcode and visible in the sidebar. It displays cumulative estimated token savings and latest-generation estimated savings.

TODO & Why i haven't made a pull request yet:

Adding real exact tokenization num values

Current measured savings

The statistics are recorded in:

~/.jcode/interlang-stats.jsonl

Token counts are approximate pre-provider estimates using chars / 4. They are not exact tokenizer counts, but they are useful for trend tracking and rough cost/context-savings estimates.

Metric	Value
Total interlang events recorded	66
Events with compression	66
Events with zero savings	0
Total estimated tokens saved	60,063
Average saved per event	910.05
Average saved per compressed event	910.05
Median saved per compressed event	744.0
Average saved over recent 20 events	1,291.95
Latest generation saved	1,587

Latest raw stats entry

{
  "blocks_encoded": 2,
  "encoded_chars": 25851,
  "encoded_tokens_estimate": 6463,
  "note": "token counts are approximate pre-provider estimates using chars/4",
  "original_chars": 32200,
  "original_tokens_estimate": 8050,
  "saved_chars": 6349,
  "saved_tokens_estimate": 1587,
  "timestamp_ms": 1777368165330
}

What llm-interlang is

The source project describes a communication layer that replaces verbose natural language with compact symbolic instructions to reduce token usage, speed responses, and increase context efficiency. Its README emphasizes these layers:

Layer	Purpose	Example
Protocol bootstrap	Teach model compact grammar	. state :mode=interlang -> ack
Predicate compression	Shorten long verbs/predicates	synchronize_state_with_cluster -> 68
Argument compression	Shorten named args	:state=x :mode=y -> :s=x :m=y
Reference compression	Reuse prior tokens/phrases	execute validate execute validate -> $1 $2 $1 $2
Pattern optimization	Collapse repetition	execute validate ; execute validate -> execute validate *2
Drift detection	Detect when model leaves protocol	response not starting with . triggers correction
Reinforcement/scoring	Measure compression ratio	English tokens vs interlang tokens

The full upstream concept is a symbolic protocol loop, not just a text compressor.

How I integrated it into Jcode

I implemented a conservative-to-verified Rust integration inside Jcode rather than directly embedding the Python bridge runtime. The key module is:

src/interlang.rs

The send-path call sites are:

src/agent/turn_loops.rs
src/agent/turn_streaming_broadcast.rs
src/agent/turn_streaming_mpsc.rs

Those are the places where Jcode prepares messages before sending them to the provider. Before the provider request is made, Jcode runs:

let (encoded, stats) = crate::interlang::maybe_compact_messages(base_send_messages);
crate::interlang::record_stats(stats);

If blocks are compacted, the outgoing message slice is replaced with the compacted version, and the dynamic system prompt gets the interlang decoder/protocol reminder.

Architecture

flowchart TD
    A[User / tools produce messages] --> B[Jcode builds outbound message list]
    B --> C{Interlang enabled?}
    C -- no --> H[Send original messages to provider]
    C -- yes --> D[Scan Text and ToolResult blocks]
    D --> E[Encode repeated lines and path prefixes]
    E --> F{Encoded form smaller enough?}
    F -- no --> H
    F -- yes --> G[Replace block with <il:v1> encoded block]
    G --> I[Append decoder/protocol reminder]
    I --> J[Record stats JSONL]
    J --> K[Send compacted request to provider]
    K --> L[Sidebar reads stats and displays savings]

<il:v1> block format

The current protocol uses self-describing blocks. Example:

<il:v1>
@p1=/home/dad/Projects/jcode-current-src/src/agent
@1=TRACE repeated diagnostic line with enough length to compress
--
$p1/turn_streaming_mpsc.rs:101: WARN ...
$1*20
</il>

Symbol table

Syntax	Meaning
<il:v1>	Start of interlang encoded block
</il>	End of interlang encoded block
--	Separates definitions from body
@N=<text>	Defines repeated line reference
$N	Expands to line reference @N
$N*COUNT	Expands to reference repeated COUNT times on separate lines
@pN=<path-prefix>	Defines a repeated path prefix
$pN	Expands to path prefix @pN

Current operating mode: verified

The default mode was changed from safe to verified.

Mode	Behavior	Risk	Savings potential
off	No compaction	none	none
safe	Self-contained <il:v1> blocks only	very low	moderate
verified	Self-contained blocks plus stronger protocol reminder and larger prefix dictionary	low	higher
aggressive	Highest prefix limits and stronger protocol reminder	medium	highest

Current mode label in sidebar:

↯ IL verified · saved ~N tok
  last gen saved ~M tok

Verified protocol safety model

Verified mode tries to make symbolic protocol use safer by making Jcode remain the source of truth. The external model is not trusted to invent or guess references.

The prompt reminder tells the model:

Jcode interlang verified protocol is active.
Decode any <il:v1> blocks before reasoning.
References are defined by Jcode and may be reused consistently across turns when present.
If any reference is unclear, say exactly `. err need_ref <name>` instead of guessing.
Treat decoded text exactly as the original message/tool output.

This gives us a path toward cross-turn symbolic protocol while keeping a fallback behavior: unknown references should be explicitly rejected, not hallucinated.

Compression algorithms implemented

1. Repeated-line encoding

Parameter	Value
Minimum text chars	900
Minimum saved chars	240
Minimum repeated line length	24
Minimum repeat count	3
Maximum repeated line definitions	32
Required compression improvement	encoded length <= 80% of original

Algorithm:

1. Split text into lines.
2. Count repeated trimmed lines with length >= 24.
3. Select highest-value repeated lines by line_length * count.
4. Emit @N=<line> definitions.
5. Replace repeated runs with $N or $N*COUNT.
6. Keep encoded block only if it clears savings thresholds.

2. Path-prefix encoding

Parameter	Safe	Verified/Aggressive
Minimum text chars	900	900
Minimum prefix length	16	16
Minimum occurrences	3	3
Maximum prefix definitions	16	48
Required compression improvement	encoded length <= 90% original	encoded length <= 90% original

Algorithm:

1. Scan whitespace tokens.
2. Detect absolute path-like tokens beginning with / or ~/.
3. Extract parent path prefix before last /.
4. Count repeated prefixes.
5. Select high-value non-nested prefixes.
6. Emit @pN=<prefix> definitions.
7. Replace every occurrence of the prefix with $pN.

Stats implementation

Stats are represented by InterlangStats:

pub struct InterlangStats {
    pub blocks_encoded: usize,
    pub original_chars: usize,
    pub encoded_chars: usize,
}

Derived values:

saved_chars = original_chars - encoded_chars
original_tokens_estimate = ceil(original_chars / 4)
encoded_tokens_estimate = ceil(encoded_chars / 4)
saved_tokens_estimate = original_tokens_estimate - encoded_tokens_estimate

Every interlang-enabled turn now records a JSONL stats event, including zero-savings turns. That means the sidebar's last gen saved line updates even when a tiny prompt does not compact anything.

JSONL schema:

Field	Meaning
timestamp_ms	Unix timestamp in milliseconds
blocks_encoded	Number of content blocks compacted this turn
original_chars	Original chars in encoded blocks
encoded_chars	Encoded chars after interlang rewrite
saved_chars	Difference between original and encoded chars
original_tokens_estimate	Approx original token estimate
encoded_tokens_estimate	Approx encoded token estimate
saved_tokens_estimate	Approx token savings
note	Reminder that estimates use chars/4

Sidebar integration

I patched the model info widget:

src/tui/info_widget_model.rs
src/tui/info_widget.rs

The widget reads aggregate status from:

crate::interlang::status_json()

It displays two lines under the model/provider details:

Sidebar line	Meaning
↯ IL verified · saved ~N tok	Cumulative estimated savings across JSONL stats
last gen saved ~M tok	Latest turn/generation estimated savings, including 0

Height reservation was updated so the model widget has enough space for both lines.

End-to-end self-test performed

I generated a synthetic path-heavy tool output:

Test property	Value
Lines generated	120
Distinct files	3
Expected WARN lines	36
Expected ERROR lines	12
Repeated prefix	/home/dad/Projects/jcode-current-src/src/agent

Interlang encoded the tool output using a path-prefix definition:

@p1=/home/dad/Projects/jcode-current-src/src/agent
$p1/turn_streaming_mpsc.rs
$p1/turn_streaming_broadcast.rs

The model still recovered the expected facts:

Fact	Expected	Result
WARN count	36	36
ERROR count	12	12
Distinct files	3	3

The synthetic test saved about:

Metric	Value
Blocks encoded	2
Original chars	32,200
Encoded chars	25,851
Saved chars	6,349
Estimated tokens saved	1,587

Limitations

Limitation	Explanation
Token estimate is approximate	Uses chars / 4, not provider tokenizer
Not full upstream protocol yet	I implemented safe/verified reversible compression, not the entire Python bridge lifecycle
Cross-turn dictionary is not fully persistent yet	Verified prompt prepares for it, but current encoding remains mostly self-contained
Savings depend on content	Tiny messages save 0; repetitive logs/paths save much more
Provider behavior varies	Some providers resend context, some use server-side session state/cache

Recommended future work

Feature	Benefit
Exact tokenizer integration	More accurate savings per provider/model
Per-session dictionary store	Larger cross-turn savings
Dictionary hash/checksum	Safer persistent reference reuse
Automatic resync on . err need_ref	Robust verified protocol recovery
Fallback resend uncompressed	Prevent model confusion from hurting task quality
Distinct stat counters per session	Cleaner sidebar metrics
Config UI for mode switching	Toggle off/safe/verified/aggressive without env vars

Practical interpretation

For normal short chat turns, savings may be zero. For coding-agent workloads, the savings can be substantial because tool outputs often contain repeated:

• absolute paths
• file names
• diagnostic lines
• stack traces
• grep/read output structure
• build/test logs
• repeated status boilerplate

Current observed average from stats:

~910.05 tokens saved per event

Recent path-heavy turns have been closer to:

~1,291.95 tokens saved per event

That means this integration is most valuable in long coding sessions where context contains lots of tool output and repeated workspace paths.

Rollback note

Earlier rollback metadata was preserved under:

/home/dad/.jcode/builds/rollback/pre-interlang-stable-target

The current stable symlink points at the interlang build tree under:

/home/dad/.jcode/builds/versions/

Start with src/interlang.rs and the send-path call sites in src/agent/turn_loops.rs, src/agent/turn_streaming_broadcast.rs, and src/agent/turn_streaming_mpsc.rs; then inspect the stats flow and the sidebar files src/tui/info_widget_model.rs and src/tui/info_widget.rs. Use the documented synthetic path-heavy self-test as a baseline, and consider the work done when stats report exact provider/model token counts rather than chars/4 estimates.